One small step for ffmpeg... one giant leap for Emby! (Looking for C developer(s) to help with transcode throttling)

[ebr 15665]

And this will keep the server from being able to start so, if you have problems with the server starting, go look for ffmpeg processes.

[CBers 7114]

And this will keep the server from being able to start so, if you have problems with the server starting, go look for ffmpeg processes.

Yep, had that happen to me earlier.

[jluce50 122]

ah yes, because it's not going to be able to respond to the q key to close. so that will need to be looked at.

 

Makes sense. Btw, when I get a little time I'm going to setup my laptop to dual-boot linux. Building ffmpeg should be much easier that way.

 

p.s. I submitted a pull request on the dev branch to add CloseHandle() after suspend/resume as recommended by MS. It's my first time using git, so hopefully I did it right...

[jluce50 122]

Quick update. I got a linux vm up and running and I was able to build a working ffmpeg executable. I made some changes to pause/unpause with keypresses and I'm now waiting on the build to complete so I can test. I'll keep you all posted...

[jluce50 122]

Aaand... it works! Pressing 'p' successfully pauses transcoding and 'u' (or any other key) unpauses. There are only a few libraries included, so the functionality is a bit limited. If anyone has access to a linux box and wants to test, just let me know and I'll upload the executable.

There is one thing that may prevent them from merging the change as-is, though. The cl output continues to update while paused (fps drops until it reaches 0). This makes sense, but they may prefer that fps freezes until unpaused, is replaced with "paused", or some other kind of indicator to the user.

Also, I just did a simple avi -> mpg conversion. ffmpeg is a massive swiss army knife of functionality and I can't even begin to test everything. I wouldn't even know where to begin. They do have a regression test suite, but I spent a couple hours trying to get it to run and was unsuccessful.

I wanted to get some input from the community (especially @@Luke ) to determine the next step. We can try to polish the functionality a bit more. I can try to get a Windows build so more people can test, possibly with MB (though no guarantees). Or I can go ahead and submit a pull request now and hope for the best. Thoughts?

Edited March 5, 2015 by jluce50

[Luke 40077]

nice!! can you at least get a log entry to indicate it was paused? i would start with that then submit it. probably good to submit sooner rather than later because we have no idea how long it will take them to even look at it.

[jluce50 122]

nice!! can you at least get a log entry to indicate it was paused? i would start with that then submit it. probably good to submit sooner rather than later because we have no idea how long it will take them to even look at it.

 

That shouldn't be too difficult (famous last words!). I'll see what I can do...

[Carlo 4552]

jluce50 can you run it redirecting the output so we can see what keeps going after it pauses.  I can maybe see this causing a problem, but it depends on what it looks like.

 

Carlo

[jluce50 122]

Sure. I'm recompiling with the logging changes now. I'll give it a shot once it's done.

[jluce50 122]

nice!! can you at least get a log entry to indicate it was paused? i would start with that then submit it. probably good to submit sooner rather than later because we have no idea how long it will take them to even look at it.

 

 

jluce50 can you run it redirecting the output so we can see what keeps going after it pauses.  I can maybe see this causing a problem, but it depends on what it looks like.

 

Carlo

 

Logging cause some weird behavior (the log text shows briefly and then disappears and the transcoding appears to continue for some reason). I changed it to write to stderr and it works much better. The fps continues to decrement while paused, but I think the "Transcoding paused" message will hopefully make it clear enough what's going on.

 

I tried redirecting the output to a file, but that doesn't accurately represent what you see on the cl. In the file there are no entries between "Transcoding paused" and "Transcoding unpaused", while the cl continues to show output while paused.

 

Here's my representation of what it looks like currently:

frame= 397 fps=202 q=3.9 size= 1338kB time=00:00:17.10 bitrate= 640.9kbits/s dup=1 drop=0
frame= 497 fps=198 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0
Transcoding paused (press 'u' to unpause)                                                   <=== new line
frame= 497 fps=198 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0   <=== new line
frame= 497 fps=180 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0
frame= 497 fps=168 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0
frame= 497 fps=159 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0
frame= 497 fps=145 q=7.1 size= 1726kB time=00:00:21.25 bitrate= 665.3kbits/s dup=1 drop=0
Transcoding unpaused                                                                        <=== new line
frame= 554 fps=160 q=16.8 size= 1948kB time=00:00:23.65 bitrate= 674.6kbits/s dup=1 drop=0  <=== new line
frame= 662 fps=177 q=23.2 size= 2324kB time=00:00:28.14 bitrate= 676.5kbits/s dup=1 drop=0
frame= 759 fps=184 q=31.0 size= 2642kB time=00:00:32.19 bitrate= 672.2kbits/s dup=1 drop=0
frame= 865 fps=191 q=31.0 size= 2942kB time=00:00:36.61 bitrate= 658.2kbits/s dup=1 drop=0
frame= 957 fps=191 q=31.0 size= 3302kB time=00:00:40.45 bitrate= 668.7kbits/s dup=1 drop=0
frame= 1055 fps=191 q=31.0 size= 3610kB time=00:00:44.55 bitrate= 663.7kbits/s dup=1 drop=0

Edited March 5, 2015 by jluce50

[Luke 40077]

hmmm....that's tricky because i'm sure there are lots of tools out there that parse the log files (including us). so after looking at that, i'm inclined to say don't do that.

[Luke 40077]

i guess i would just leave the  log alone

[jluce50 122]

i guess i would just leave the  log alone

 

With my example above, I'm not logging, just writing to stderr. I was showing what appears on the cl, not in the log. Writing to stderr doesn't affect the log, does it?

[Luke 40077]

yea stderr is actually where the log goes

[jluce50 122]

Well shoot, okay. I'm worried that without some indication to the user that transcoding is paused they're not as likely to accept the change...

 

For what it's worth, printing to stderr looks to be standard procedure for providing feedback from keyboard interaction.

Edited March 5, 2015 by jluce50

[Luke 40077]

i would say keep the log writing as it is, maybe add paused=1 to the end of the line so that the formatting stays the same. but obviously don't add paused=0 the rest of the time because i don't think people are going to want to see that

[Carlo 4552]

I would think they would want the timer stopped and no output.  Then when it resumes the timer is again started and output continues.

 

The "timer" is going to be a bit trickier if required.

 

But I would think most people are going to want to know the FPS of only time actually spent processing the file and not taking the paused time into affect which would give bogus low numbers.

[FredipusRex 12]

Aaand... it works! Pressing 'p' successfully pauses transcoding and 'u' (or any other key) unpauses. There are only a few libraries included, so the functionality is a bit limited. If anyone has access to a linux box and wants to test, just let me know and I'll upload the executable.

There is one thing that may prevent them from merging the change as-is, though. The cl output continues to update while paused (fps drops until it reaches 0). This makes sense, but they may prefer that fps freezes until unpaused, is replaced with "paused", or some other kind of indicator to the user.

 

Cool! Congratulations. I think you just need to do a "paused" check before the print_report() call in transcode(), e.g.

 

++ if (!paused)

print_report(0, timer_start, cur_time);

[jluce50 122]

Cool! Congratulations. I think you just need to do a "paused" check before the print_report() call in transcode(), e.g.

 

++ if (!paused)

print_report(0, timer_start, cur_time);

Yeah, that's what I ended up doing. I had to stop for the day before it finished building, though. I'll be able to test it in the morning...

[FredipusRex 12]

Actually, I think a better method would be to track elapsed "pause" time. It requires a few more variables but it prevents some misleading statistics.

 

I'm not by a keyboard (iPad at the moment) so I can't create a pseudo-diff, but the basic concept is this:

 

1. Create two new file-scope int64_t variables, pausedTime and pauseStart. Set them to 0.

2. When the paused flag is set, also set pauseStart = av_gettime_relative();

3. When the paused flag is unset, check if pauseStart != 0 { pausedTime += (av_gettime_relative() - pauseStart); pauseStart = 0;}

4. Create a function called gettime_relative_minus_pause() {return av_gettime_relative() - pausedTime - (pauseStart ? av_gettime_relative() - pauseStart : 0); }

5. In transcode(), change the int64_t cur_time = av_gettime_relative(); to int64_t cur_time = gettime_relative_minus_pause();

6. Also, in transcode(), change print_report(1, timer_start, av_gettime_relative()); to print_report(1, timer_start, gettime_relative_minus_pause());

 

What we're doing here is effectively pausing "the clock" by subtracting away any time that has or is being spent in a pause state. The first 3 steps track pause time. The fourth step gets "now" but subtracts away time spent in the pause. The final two steps use that "now - pause" time stamp for the current time, which will make print_report think that nothing is happening while during a pause and prevent a printout.

 

This is a better solution than the original idea because the original version will have ridiculously long "transcode" times on the first log entry after a pause - the code doesn't know its been idling. By modifying what "now" is, we're letting the existing code "know" about the pause in a backward compatible way that only touches two existing lines of code.

[Carlo 4552]

FredipusRex, exactly the type of thing I think will be needed to get this accepted.

[gillmacca 17]

Using the latest dev build, I am noticing random skipping when playing video files. In a typical episode, it is happening 2 or 3 times. Not sure of the exact length of the skip, but I would guess about half a second or so

Anyone else seeing this? It's only happened since the update to 'adjust process throttle'

[FredipusRex 12]

FredipusRex, exactly the type of thing I think will be needed to get this accepted.

I think you need to do minimal changes to get a pull request accepted, so I agree. In fact, I could argue to get rid of the original "paused" flag and use pauseStart for that purpose (it would work and remove another change).

[jluce50 122]

I think you need to do minimal changes to get a pull request accepted, so I agree. In fact, I could argue to get rid of the original "paused" flag and use pauseStart for that purpose (it would work and remove another change).

 

I'll incorporate that into what you suggested above and see what I can do. I had forgotten how frustrating it is to have to wait 20 minutes for a build every time you want to test a change, but hopefully I'll have something later this morning. Thanks!

[jluce50 122]

Anyone else seeing this? It's only happened since the update to 'adjust process throttle'

 

Nope, I watched a bunch of stuff on my Roku last night and didn't notice a single hiccup. Just out of curiosity, what processor are you using on your MB3 server?