One small step for ffmpeg... one giant leap for Emby! (Looking for C developer(s) to help with transcode throttling)

[Carlo 4330]

I played around and both methods worked fine for me on Windows to pause ffmpeg.

 

I'm liking this solution myself because it SHOULD JUST WORK with no grief compared to using -ss and -t commands with ffmpeg which could be much more prone to errors with different codec or builds of ffmpeg (much more to test, ect).

 

It should be simple to come up with a clean library to pause/resume as well as change priority and/or cores/affinity of any process. So between the two mechanisms we should be able to handle this well.

 

It's now "to simple" so I'm gonna throw a monkey wrench into things.

 

We have a side project in the works that is going to be useful for both distributed encoding and hopefully soon "GPU" assisted transcoding.   In a nutshell it's a "proxy or intercept" EXE.  So it disguises itself as ffmpeg.exe (on windows) and receives the commands on the command line from MB3.  The intercept then in turn calls/runs ffmpeg on the same or different computers.  As far as MB3 is concerned it thinks IT IS talking to ffmpeg but instead is talking to the "man in the middle".

 

So we can't just suspend the "ffmpeg.exe" process because it could end up being the intercept program and not ffmpeg itself.  In this case the real ffmpeg would continue to run but the "communication layer" gets shut down messing everything up.

 

So ideally, we should try and keep this in mind from the start.  My point is not to just assume the process we called directly from MB3 is the one we want to SUSPEND/RESUME.

[Carlo 4330]

A couple of semi-easy ways to handle this could be a config option in admin:

 

Using ffmpeg directly? (default to Yes) or a list box similar to Method to interact with ffmpeg (direct, proxy).

 

If the default of direct is choosen then MB3 can suspend/resume the ffmpeg exe directly as has been talked about.

If the method is "proxy" then MB3 doesn't try to control ffmpeg but instead just updates an array (or similar) with a status of the encode.

 

So for example with the proxy it could keep track of the process using the TEMP transcode name which are always unique.  It then just puts the status it wants the proxy to perform such as "suspend", "resume" or "low", "medium", "high" for priority changes.  It makes this available via an API or simply updates a JSON or XML file.  With an API the proxy calls the API with the TEMP file name and gets back a status (ie resume).  It then performs the action.

 

So in one case MB3 performs the actions itself and in the other case it just updates a list of transcodes which the proxy pulls from.  It could pull this info every 5 to 15 seconds which we can test to figure what works best.

 

That should be pretty simple and would FUTURE PROOF us right from the start.

 

Carlo

 

PS to get started we start ASSUMING it set to "direct" to keep things simpler and get the suspend/resume thing working and tested.  Then once comfortable with this add the option of direct vs proxy. WIN/WIN

Edited February 26, 2015 by cayars

[Carlo 4330]

Was thinking of a couple of other things.  The "intercept/proxy" program will probably be a good thing to have even if not doing GPU or distributed encoding.  We are now starting to get plugins that have the ability to call ffmpeg directly such as the Roku BIF (trick play) functionality.  There will also be a universal remuxer/transcoder I'll be adding shortly and these will be competing for the use of ffmpeg.  So by being able to use the intercept program it would have the ability to manage some of these "extra" ffmpeg processes and treat them as "low" or background processes while being able to keep the real-time encodes as "foreground" or higher important processes.

 

Without this, we could run into the situation of trying to suspend/resume 3 or 4 real-time transcodes but end up competing with these "extra" processing monopolizing the CPU.

 

Carlo

 

PS sorry about all the posts in a row, but as I'm thinking of these different things/scenarios that will come into play, I'm trying to keep them all in consideration and "document" them so we don't forget about them and how they can affect the overall process we are trying to achieve.  Because ultimately we want to be able to control ALL the ffmpeg processes running on the machine and have a decent way of deciding what to consider low importance vs high importance for the CPU(s) at any given time.

Edited February 26, 2015 by cayars

[jluce50 118]

So we can't just suspend the "ffmpeg.exe" process because it could end up being the intercept program and not ffmpeg itself.  In this case the real ffmpeg would continue to run but the "communication layer" gets shut down messing everything up.

 

Using ffmpeg directly? (default to Yes) or a list box similar to Method to interact with ffmpeg (direct, proxy).

 

If the default of direct is choosen then MB3 can suspend/resume the ffmpeg exe directly as has been talked about.

If the method is "proxy" then MB3 doesn't try to control ffmpeg but instead just updates an array (or similar) with a status of the encode.

 

Was thinking of a couple of other things.  The "intercept/proxy" program will probably be a good thing to have even if not doing GPU or distributed encoding.  We are now starting to get plugins that have the ability to call ffmpeg directly such as the Roku BIF (trick play) functionality.  There will also be a universal remuxer/transcoder I'll be adding shortly and these will be competing for the use of ffmpeg.  So by being able to use the intercept program it would have the ability to manage some of these "extra" ffmpeg processes and treat them as "low" or background processes while being able to keep the real-time encodes as "foreground" or higher important processes.

 

Unless I'm missing something, I think globally using the ffmpeg wrapper/proxy would address all these issues. Let's call it TranscodeManager. Basically, the TranscodeManager would contain all the logic for interacting with the ffmpeg instance(s). MBS (or any other consumer) wouldn't know or care about any of this. MBS would just tell TranscodeManager to 'pause' transcoding and it would either pause ffmpeg or update the array (or whatever). Any other plugins/clients that require ffmpeg would call into the same (singleton?) instance and new processes could be spun up at the appropriate priority level. Does that sounds like what you're thinking?

[Carlo 4330]

There are plusses and minumsses to each approach.

 

If MB3 controlled the instances (when it can) directly there would not be any delay and it would be "finer grained".  I would think this would also make it easier to test and to get to release quicker.  There would also be no no reliance on a third party for the principle operations.  The downside is there is more work for Luke (but not much).

 

Going the route of feeding everything through the TranscodeManager would make life easier for Luke for the initial programming but then would add a layer of diagnostics when something isn't working correctly.  At least if there was an "Off", "Direct Control" and "Transcode" option for how this is handled we would have more ways to diagnose problems if/when the happen which could be preferred.

 

Certainly a good reason to funnel everything through the TranscodeManager is that all the operating specific mechanisms to pause and change priority are all done in one set of code and there could be different versions of the TranscodeManager for different operating systems which is going to be needed most likely. So the operating system specific issues are moved outside the core of MB3.

 

So 6 of one and half dozen of another.

 

Pretty much Luke's call I'd think.

Carlo

[jabbera 23]

Plex ffmpeg source code is available here: http://files.plexapp.com/elan/ffmpeg/PlexNewTranscoder.tar.bz2

 

Look at what they do.

[Luke 37066]

I think we're good for now, thanks. We have a route to explore that potentially doesn't require our own ffmpeg build. So right now i prefer that unless we discover that it won't be feasible.

[jluce50 118]

Just sharing in case anyone is interested. Turns out Plex throttles by using a simple formula to calculate sleep time in the input processing loop, rather than altering the read fps (or some other means). Rather than passing in additional commands via the command line, they make http calls to the server directly from the processing loop (why didn't I think of that?!). They've got a ton more custom functionality that I haven't even begun to dig into as well. They've tweaked ffmpeg pretty heavily to handle stuff like subtitle decoding, seeking, segmenting, etc. I've been curious about this for a long time, so it's kinda cool to see how it actually works...

Edited February 26, 2015 by jluce50

[Luke 37066]

I personally would rather keep the encoder dumb and allow it to be controlled externally, but hey if it works.

 

Why don't we do twi things.

 

Can you add a pause command and submit a pull request to ffmpeg?

 

We can play with suspension while we wait for it.

[jluce50 118]

I can give it a shot, but I'd feel much more comfortable if I had some folks familiar with C looking over my shoulder. 

 

Do you prefer a full-on pause to a "delay multiplier" or something along those lines to slow down processing?

[Luke 37066]

Start with pause because I think that could have value to other products using ffmpeh so I don't see why they wouldn't accept it. You can expand on it later. You might just need a paused flag then key inputs to set it.

[Carlo 4330]

I doubt they will consider it since ffmpeg doesn't allow re-entry and just starts a new instance with every new command line call.  It would/could change the functionality of ffmpeg and break a lot of existing things.

 

You never know of course and it never hurts to ask but I wouldn't hold my breath.

 

Carlo

[Carlo 4330]

Luke you never said (or I missed it) but are you going to try and do the suspend/resume directly from MB3 at least for phase 1 implementation?

[Luke 37066]

what would it break? its not re-entry it's just having it call thread.sleep until told to resume

[Luke 37066]

Luke you never said (or I missed it) but are you going to try and do the suspend/resume directly from MB3 at least for phase 1 implementation?

yes

[Carlo 4330]

what would it break? its not re-entry it's just having it call thread.sleep until told to resume

 

What I meant is that ffmpeg at present doesn't allow you to interact with it from the command line once it's running. Any new commands on the command line would just use a new instance of ffmpeg (far as I know). It could however watch for a keyboard combination and be able to use that if it has "focus".  I wasn't thinking of the 2nd way just mentioned when I posted.

 

I've never tried but it's possible the PC "pause key" might do this already.  I'd try it but I'm remote at the moment.

[Luke 37066]

it accepts the 'q' key over stdin to exit. that's how we stop it instead of brute force killing the process. so my thought was we could just add more keys for more commands

[jluce50 118]

Just fyi, it actually accepts a handful of keys:

+            if (key == '?'){
+                fprintf(stderr, "key    function\n"
+                                "?      show this help\n"
+                                "+      increase verbosity\n"
+                                "-      decrease verbosity\n"
+                                "h      dump packets/hex press to cycle through the 3 states\n"
+                                "q      quit\n"
+                                "s      Show QP histogram\n"
+                );

[Luke 37066]

ah very nice. looks like you've gotten to know it quite well now

 

definitely add pause/unpause, if you're comfortable enough and want to add lower speed rates you can go ahead and do that too. that will allow us to do things like different speeds for playback and sync transcoding. for sync we just use -re but that ends up being super slow.

[jluce50 118]

ah very nice. looks like you've gotten to know it quite well now

 

definitely add pause/unpause, if you're comfortable enough and want to add lower speed rates you can go ahead and do that too. that will allow us to do things like different speeds for playback and sync transcoding. for sync we just use -re but that ends up being super slow.

 

I don't know about that! C is still mostly greek to me. I can make sense of stuff at a high level, but actually getting in there and making changes will be very slow going.

 

I'm assuming you'd want to submit a pull request for the throttling too? I'm not very confident they'd accept that change. The implementation will be somewhat arbitrary since I don't know of a good way to throttle by percentage and we can't pass parameters with the key press. I doubt they'll see a broad enough appeal to include it, but who knows...

[Luke 37066]

ok let's just do pause then. even that by itself is a huge win for us.

[ebr 14913]

Don't forget unpause too...

[jluce50 118]

Don't forget unpause too...

 

Here we go with the nitpicking... ;-)

[Carlo 4330]

Don't forget unpause too...

That just might come in handy!

[Carlo 4330]

So you ready for us to start testing this yet Luke? LOL