Poor quality when transcoding and time scrubbing

[Waldonnis 148]

Hi again,

 

in the meanwhile I had a closer look at the transcoding log. I'm not that familiar with ffmpeg, but I wonder whats going on in the first 1000 frames:

frame=  153 fps=0.0 q=32.0 size=N/A time=00:51:21.27 bitrate=N/A speed=6.16e+03x    
frame=  311 fps=310 q=25.0 size=N/A time=00:51:27.59 bitrate=N/A speed=3.08e+03x    
frame=  470 fps=313 q=24.0 size=N/A time=00:51:33.95 bitrate=N/A speed=2.06e+03x    
frame=  622 fps=311 q=30.0 size=N/A time=00:51:40.03 bitrate=N/A speed=1.55e+03x    
frame=  783 fps=313 q=33.0 size=N/A time=00:51:46.47 bitrate=N/A speed=1.24e+03x    
frame=  925 fps=307 q=35.0 size=N/A time=00:51:52.17 bitrate=N/A speed=1.03e+03x  

Is q=* indicating the resulting quality? Is it the same 'measurement' you would usually do with h264 by defining the target quality with CRF? So a value of 32 or even higher might indicate a very bad quality, because a good result would usually achieved with a value something between 18 and 23?

 

Thanks,

Peter

 

q is the quantizer value, which will fluctuate for a given CRF depending on the complexity of the frames/scene and isn't a fair representation of the quality of the output when looked at on its own (it's reflective of CRF, but not equivalent to it).  A sequence of black frames wouldn't require much work/analysis to achieve high quality, for example, so the quantizer value wouldn't need to be low for a given CRF.  A more difficult frame sequence, on the other hand, would require a lower quantizer value to achieve a given CRF/"quality".

[Pittiplatsch 2]

Thanks for your replies.

 

@@Waldonnis Thanks for enlighten me, sounds comprehensible to me. So the quantizer values in the beginning seems to be pretty high.The rendered frames aren't black or something like this, because I didn't start in the beginning of the movie where black frames are quite normal. So they are 'normal' frames somewhere in the middle of an ordinary movie.

 

@@Luke I did a quick test with Google Chrome (59) as well. The same here. After I tweaked the transcoding settings yesterday, the quality after seeking is restored after about 30 to 60 seconds. It is still a bit long for my taste. But after looking at the logs it seems plausible, because the period of time with poor quality are the frames renderer with q=30 or above. I think there is a connection. Any idea what could cause this behavior?

 

Thanks again,

Peter

[Waldonnis 148]

Thanks for your replies.

 

@@Waldonnis Thanks for enlighten me, sounds comprehensible to me. So the quantizer values in the beginning seems to be pretty high.The rendered frames aren't black or something like this, because I didn't start in the beginning of the movie where black frames are quite normal. So they are 'normal' frames somewhere in the middle of an ordinary movie.

 

@@Luke I did a quick test with Google Chrome (59) as well. The same here. After I tweaked the transcoding settings yesterday, the quality after seeking is restored after about 30 to 60 seconds. It is still a bit long for my taste. But after looking at the logs it seems plausible, because the period of time with poor quality are the frames renderer with q=30 or above. I think there is a connection. Any idea what could cause this behavior?

 

Thanks again,

Peter

 

I'm not sure.  ffmpeg/x264 should be producing output per the command line options immediately and consistently throughout in this situation.  A quality drop when starting from an arbitrary point in a file is entirely possible depending on how the source file was encoded, but it shouldn't last longer than a couple of seconds at most (depending on the GOP length).  The maxrate being used is pretty generous, considering the source's bitrate, so I doubt that's affecting the quantizer values too much.  The source file's bitrate is quite low even for a 720p source, though, so I wouldn't expect too much out of it (for comparison, DVD max bitrate is 9.8Mb/s and most streaming sites tend to cap out at a quarter of that).

 

What I'm not sure of is how the subtitle overlay is affecting the encoder and output.  Subtitle overlays often make scenes more difficult since they introduce hard edges, as well as making motion estimation harder (halo artifacts are sometimes noticed in these cases).  I don't think it shouldn't have too great an effect on the bitrate such that it greatly increases the chances that a given scene will exceed the bitrate limit (only on how hard the encoder has to work)...unless they got crazy with them and used subtitle streams for effects (I have seen this, believe it or not) or are more complex.  Those types of subtitle image uses are quite rare, though.

 

It's a puzzle, that's for sure.  If a manual transcode with the same settings seemingly differs in quality, then I'm just not sure what else to look at outside of maybe the browsers or video drivers (if acceleration is used by the browsers).  On a side note, if burning in subtitles is the only time you really need to transcode and this continues to be a problem, then it may be worth finding SRT subs for your files (or converting them yourself), since nearly everything can handle those without having to burning them in.  Unfortunately, doing so is really not an option for some languages...and some anime subs would be nigh impossible to convert, so there are cases where this wouldn't be possible (and SRT is limiting when it comes to formatting/layout).

[Pittiplatsch 2]

I'm not sure.  ffmpeg/x264 should be producing output per the command line options immediately and consistently throughout in this situation.  A quality drop when starting from an arbitrary point in a file is entirely possible depending on how the source file was encoded, but it shouldn't last longer than a couple of seconds at most (depending on the GOP length).  The maxrate being used is pretty generous, considering the source's bitrate, so I doubt that's affecting the quantizer values too much.  The source file's bitrate is quite low even for a 720p source, though, so I wouldn't expect too much out of it (for comparison, DVD max bitrate is 9.8Mb/s and most streaming sites tend to cap out at a quarter of that).

 

What I'm not sure of is how the subtitle overlay is affecting the encoder and output.  Subtitle overlays often make scenes more difficult since they introduce hard edges, as well as making motion estimation harder (halo artifacts are sometimes noticed in these cases).  I don't think it shouldn't have too great an effect on the bitrate such that it greatly increases the chances that a given scene will exceed the bitrate limit (only on how hard the encoder has to work)...unless they got crazy with them and used subtitle streams for effects (I have seen this, believe it or not) or are more complex.  Those types of subtitle image uses are quite rare, though.

 

It's a puzzle, that's for sure.  If a manual transcode with the same settings seemingly differs in quality, then I'm just not sure what else to look at outside of maybe the browsers or video drivers (if acceleration is used by the browsers).  On a side note, if burning in subtitles is the only time you really need to transcode and this continues to be a problem, then it may be worth finding SRT subs for your files (or converting them yourself), since nearly everything can handle those without having to burning them in.  Unfortunately, doing so is really not an option for some languages...and some anime subs would be nigh impossible to convert, so there are cases where this wouldn't be possible (and SRT is limiting when it comes to formatting/layout).

 

Hi,

 

I'm sorry for the delay.

 

Yeah, the source footage could cause this, but I don't use fancy settings here for encoding my stuff. It's based on x264 with High Profile and the basic 'Slow' preset.

 

You are actually right, using text based subtitles might be a solution. I was used to do conversions a while ago, but honestly, it's a lengthy process, even with the good OCR tools out there. Getting srt's from the Web isn't working for me that good as well. I also tried Emby's build-in tool for this. The problem is, I mostly rely on SD material and everybody else seems to be focused on HD these days. Since the framerate is different, I have to convert this manually. It's possible, but also a time consuming process. Additionally there are sometimes big editorial differences between DVD and Bluray releases, so these subtitles won't match anyway.

 

I played around with ffmpeg in the last couple of days. I did use the CLI command from the log as a starting point and this time I could reproduce the issue. I mean I'm getting the same high quantizer results and the appropriate poor quality in video playback in the beginning. I also removed the subtitle burn-in from the ffmpeg option. Same here. What I figured out is this ffmpeg option might cause some trouble:

-force_key_frames "expr:gte(t,n_forced*3)"

I removed this and voila, the quality shift in the beginning isn't appearing anymore. I also played around with less forced keyframes (n_forced*20) and with this the high quantizer results are lasting only a couple of seconds.

 

 

Note that forcing too many keyframes is very harmful for the lookahead algorithms of certain encoders: using fixed-GOP options or similar would be more efficient.

 

This is taken from the ffmpeg documentation. It's seems there is really a connection to the issue I have. 

 

Is there a way for me having more control about ffmpeg when Emby is calling it?

 

Thanks,

Peter

[Luke 37994]

With HLS transcoding we are following HLS guidelines of at least one keyframe per segment. Reducing that will bring us out of spec and may cause other problems.

[Pittiplatsch 2]

With HLS transcoding we are following HLS guidelines of at least one keyframe per segment. Reducing that will bring us out of spec and may cause other problems.

 

Ok, I understand.

 

Any other suggestions so far?

[Waldonnis 148]

You are actually right, using text based subtitles might be a solution. I was used to do conversions a while ago, but honestly, it's a lengthy process, even with the good OCR tools out there. Getting srt's from the Web isn't working for me that good as well. I also tried Emby's build-in tool for this. The problem is, I mostly rely on SD material and everybody else seems to be focused on HD these days. Since the framerate is different, I have to convert this manually. It's possible, but also a time consuming process. Additionally there are sometimes big editorial differences between DVD and Bluray releases, so these subtitles won't match anyway.

 

Yeah, I totally understand.  I do manual subtitle conversion for everything and it's definitely not a fast process...doubly so for DVD subtitles, since most of it is much lower resolution (and harder for OCRs to get right).  You can correct the timing of most BD-sourced SRT subs rather easily, but I agree that it's still not a good/fun solution since I've seen quite a few that require a lot of hand-holding.

 

I played around with ffmpeg in the last couple of days. I did use the CLI command from the log as a starting point and this time I could reproduce the issue. I mean I'm getting the same high quantizer results and the appropriate poor quality in video playback in the beginning. I also removed the subtitle burn-in from the ffmpeg option. Same here. What I figured out is this ffmpeg option might cause some trouble:

-force_key_frames "expr:gte(t,n_forced*3)"

I removed this and voila, the quality shift in the beginning isn't appearing anymore. I also played around with less forced keyframes (n_forced*20) and with this the high quantizer results are lasting only a couple of seconds.

 

 

This is taken from the ffmpeg documentation. It's seems there is really a connection to the issue I have. 

 

Is there a way for me having more control about ffmpeg when Emby is calling it?

 

Thanks,

Peter

 

As Luke said, having a keyframe at the start of every segment is highly recommended (may even be required - I never really looked too deeply at that since it makes sense to do so).  Emby's telling ffmpeg to create 3sec segments, if I remember correctly...hence the expression.  It's not the only way to do it, but it's pretty commonly referenced from the documentation.

 

I have a few ideas of why this may be going on, but am going to have to run some tests locally first with roughly the same Handbrake preset you're using (thanks for providing that info, btw).  I'll try to get to it in the next few days (busy week).  If you get a chance, can you run the following command against both the source file and a resulting transcode that is showing the issue, then PM me the generated text files?  Just substitute FILE for the filename and you can change the FILE.txt to a name indicating which is which (source.txt and notworking.txt or something similar).  The resulting files may be larger than attachments allow, so you may have to zip them.

ffprobe -i FILE -select_streams v -show_frames -of csv -show_entries frame=key_frame,pict_type,pkt_dts_time > FILE.txt

To explain what that does: it dumps a few bits of info from each frame as a comma-delimited list.  key_frame is whether or not it's marked as a key_frame, pict_type is what type of frame it is (I, B, P), and pkt_dts_time is basically the timestamp of the frame (not really needed, but would be interesting to see if there's anything wrong there).  The output of it in the text file will look like this but a lot longer (this is a very short snippet from one of my BD rips):

frame,1,0.000000,I
frame,0,0.042000,B
frame,0,0.083000,P
frame,0,0.125000,B
frame,0,0.167000,B
frame,0,0.209000,P
frame,0,0.250000,B
frame,0,0.292000,B
frame,0,0.334000,P

I doubt I'll see anything unexpected, but it'll give me a better idea about the source's/output's GOPs as well as maybe offering a clue or two as to what may be going on.

[Waldonnis 148]

FYI, I'm seeing what you're seeing on my initial test using a 720p source file with subs, so you can ignore the frame dump request.  I'll see what I can make of this...

[Pittiplatsch 2]

@@Waldonnis Many thanks for your support. I appreciate it. No worries and no need for hurry, I've a busy week too and I don't think I'm able to give it another try before weekend. I also want to play around with different source material. Let's see.

 

 

FYI, I'm seeing what you're seeing on my initial test using a 720p source file with subs, so you can ignore the frame dump request.  I'll see what I can make of this...

 

So you can reproduce the issue?

[Waldonnis 148]

@@Waldonnis Many thanks for your support. I appreciate it. No worries and no need for hurry, I've a busy week too and I don't think I'm able to give it another try before weekend. I also want to play around with different source material. Let's see.

 

 

So you can reproduce the issue?

 

Yep.  I managed to get better output by removing the -force_key_frames and fixing the GOP length to roughly three times the framerate (with -g).  This should have the same effect as -force_key_frames does, but it's done in x264 itself and is very predictable.  It was a very rudimentary test, though, and I need to validate the results more thoroughly (and try it with a few other samples, along with looking more at the muxer options to make sure it all matches up).  I also need to validate that scenecut I-frame generation is being used (again, quick preliminary test, so my initial seek point wasn't well chosen to test this).  It's a rather specific solution/idea, though, so I wouldn't be too quick to recommend it just yet.

 

With the command line in your logfile, it's supposed to generate a keyframe at 3sec intervals as previously noted, to match the segment_time.  I'm not sure if it's being affected by the seek, though.  It seems to be generating a bunch of very, very small segments with only one frame in each segment (unsurprisingly marked as a keyframe) until it reaches a certain point, then starts generating normal segments.  The first few frames look great, but it rapidly goes downhill (about 12 frames in with my sample, but the degradation corresponds with a scene cut, so that adds a wrinkle).  I'm not sure what's going on with -force_key_frames here, but I've been trying a few variations on the expression and plan on doing a full logging run to see what ffmpeg thinks is going on.

[Waldonnis 148]

I think I found it.  The expression only works correctly if we started in the beginning of the file.  If we don't, n_forced starts as one (for the first keyframe inserted), while t (time) is likely significantly higher than n_forced*3.  For example, if we seek 30mins into a file, the time is going to be 1800secs, which is a lot higher than 3, so it's generating keyframes until n_forced*3 "catches up" such that it's greater than or equal to the time of a given frame.  The farther you seek, the longer that string of forced keyframes will be.

 

Here's an expression that should work better:

expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+3))

Can you replace Emby's expression with the above in your command line tests and see if it helps?

 

What it does is check to see if we've previously forced a keyframe (the isnan(prev_forced_t) part).  If not, then it checks to see if t=t, which of course it does, so it forces a keyframe.  This should happen immediately and, once we force one keyframe, prev_forced_t should contain a time from that point on.  If we did force a keyframe before, it checks to see if the current frame's timestamp is greater than or equal to the previous keyframe's timestamp plus 3 (secs) - if this is true, it'll force a keyframe and prev_forced_t will be set to that frame's time...and the process continues.

 

I really wish we had access to the exact framerate in this particular expression rather than just frame numbers and timestamps since we could be a lot more precise, but we don't.  It's also probable that we'll run into slight rounding errors with the time, since it's somewhat ham-handed approach.  I may have a way to overcome that, but I have to review ffmpeg's docs and code to see what functions are available to me.  One idea I had was to reset the timestamps after the seek and just use the original expression, but that has some side effects that might be...undesirable...so that's out.

 

I still need to verify that the expression works in all situations (seek vs. starting from the beginning) and that the output has the proper keyframe/GOP layout, so I won't recommend any changes quite yet....I just want to see if my results match up with yours.

Edited July 12, 2017 by Waldonnis

[Luke 37994]

Thanks @@Waldonnis !

[Waldonnis 148]

No problem.  A fixed GOP with a predictable keyframe interval would probably be better, but it would require passing a few more arguments to x264.  I know exactly what to do for that, but it's probably not worth looking into because hardware encoders have fewer (and different) GOP-related options.  If we were making static segments at multiple bitrates for adaptive streaming, this would be a no-brainer    I still want to try a few other ideas and verify the results with other input files as well, but can't get to it today.  There are a few potential issues with that expression as well that I want to confirm/solve.

[Waldonnis 148]

Seems that expression is working nicely and predictably.  Here's the cadence being produced from one sample (720p 29.97 fps DVD rip):

Forced keyframe at time 3075.038633
Forced keyframe at time 3078.041633
Forced keyframe at time 3081.044633
Forced keyframe at time 3084.047633
Forced keyframe at time 3087.050633

Same parameters for a 1080p 23.976 fps BD rip:

Forced keyframe at time 3075.030292
Forced keyframe at time 3078.033292
Forced keyframe at time 3081.036292
Forced keyframe at time 3084.039292
Forced keyframe at time 3087.042292

As you can see by the ffmpeg report snippets, the keyframes are being generated at periods slightly longer than 3secs (confirmed by frame dumps), which shouldn't be an issue for the segment muxer (it splits based on GOP anyway, so it'll ignore -segment_time if the GOP is longer unless you force it with other options).  I didn't get a chance to test with a 60fps file, but it should behave the same.  For grins, I removed the seek and just started transcoding/segmenting from the beginning of the files...same cadence of forced keyframes.

 

Outside of scenecuts, the segments are all ~3.0x secs long as well.  The DVD rip had a rapid sequence of scenecuts at one point, so it generated some smaller segments (~0.3ish secs in some cases).  I expected this, though, since x264 by default creates keyframes at scenecut points and we're forcing additional ones on the ~3sec mark, so the segment muxer just creates smaller segments when it hits these points (the old expression would result in the same thing happening, so this isn't different behaviour).

 

I tried a few other ideas, but they didn't work out since the variables accepted by force_key_frames are just too limited to do much else with.  I also didn't see the issues I expected to see (drift and rounding errors), so it's probably the best it's going to get.  One thing I overlooked before was that the segment_time_delta being used in the logfile was unusually large.  May want to check that it's being set to the desired value (currently seems to reflect the seek time rather than a PTS time delta for looking for keyframes to split on).

[TopSideControl 11]

Hi @@Luke, 

 

Just wondering if there was any update on this?

 

Thanks

Brian

[Luke 37994]

https://emby.media/community/index.php?/topic/48786-poor-quality-when-transcoding-and-time-scrubbing/

[TopSideControl 11]

Sorry for jumping in but I was directed here by Luke for the same issue I've been experiencing since January.

 

I appreciate you are still working on a solution but is there a rough timeframe as to how long this would take to be released in any build (dev or beta).

 

Sent from my SM-G920F using Tapatalk

[Pittiplatsch 2]

@@Waldonnis This sounds like pretty good news to me. Thank you very much for all your efforts.

 

I also tried the expression you mentioned above and it also works with the source material I'm currently using. I just watched if there are visual quality issues, I didn't do such a deep analysis like you did. Anyway, awesome work.

 

Thanks so much,

Peter

 

[TopSideControl 11]

Is there any news on when this fix might make it into a future release? Thanks. 

[Luke 37994]

@@TopSideControl thank you for your patience

[TopSideControl 11]

@@TopSideControl thank you for your patience

 Thanks Luke, I'm not trying to be a pain about this but I've been trying to get this fixed for 6 months now. 

[Pittiplatsch 2]

I don't wanna push or something like this, just curious what the status with the issue is? Any chance for getting the fix @@Waldonnis was achieving with the next release somehow?

 

Thanks,
Peter

[TopSideControl 11]

Hi Guys, Any idea when we can expect a fix for this?Thanks 

[Luke 37994]

It's being looked into, thanks.

[TopSideControl 11]

Any idea when there will be progress. Its the main reason I run emby so I can watch stuff while away from home, not to mention the reason I donate.