Jump to content

HEVC Profile Main 10 / QSV Hardware Transcoding / staircase effect


SoGX
 Share

Go to solution Solved by softworkz,

Recommended Posts

Can you show a screenshot (with the Sony logo)?

And once another (different) attempt:

  • Under "Transcoding", choose "Advanced" and then go to the H.265 (HEVC) section:
  • Uncheck all and check only "DX11VA ...." 
Link to comment
Share on other sites

With the settings, the video quality gets worse again.

The Sony logo has still been taken without the new settings

 

image.thumb.png.ca09450df85f977f47e9a5b42c7de9de.png

 

 

image.png.a19228a7e059c4136caaa6cf77a18340.png

image.png.b365eb5c390526d43e2049703cb6ff66.png

Link to comment
Share on other sites

20 minutes ago, SoGX said:

With the settings, the video quality gets worse again.

You mean that you removed the replacement to scale_qsv and chose the DX11VA decoder and now it's bad like before, right?

 

What we can try now is a combination:

  • The decoder doesn't seem to matter, so we can stick to the QuickSync one
  • Use the replacement to scale_qsv
  • Under "Codec Parameter", choose "veryslow" for the preset

Additionally you could also try different bitrate modes (after toggling "Show advanced settings")

One thing we cannot test is the combination of low_power plus scale_qsv (like JF does), because we can do just a single replacement at a time.

 

Link to comment
Share on other sites

24 minutes ago, softworkz said:
54 minutes ago, SoGX said:

With the settings, the video quality gets worse again.

You mean that you removed the replacement to scale_qsv and chose the DX11VA decoder and now it's bad like before, right?

 Yes, that's what i mean.

 

28 minutes ago, softworkz said:

What we can try now is a combination:

  • The decoder doesn't seem to matter, so we can stick to the QuickSync one
  • Use the replacement to scale_qsv
  • Under "Codec Parameter", choose "veryslow" for the preset

Additionally you could also try different bitrate modes (after toggling "Show advanced settings")

The settings now look like this. The picture is a little better but still not good. I have sent you login data, so you can have a look at the result.

 

image.png.60ff27eee142ff7df8569e6cd8f70e14.png

Link to comment
Share on other sites

Are you sure that you don't see the "artifacts" in the logo when you don't do the scale_qsv replacement?

Link to comment
Share on other sites

At the moment the settings look like this and I don't see the artefacts, but the image is very grainy.

image.png.bbfd2fc6e5b7a3cb8d0c11d56bf8c785.png

 

If I delete the options again, the picture looks like at the beginning of the thread.

Link to comment
Share on other sites

47 minutes ago, SoGX said:

If I delete the options again, the picture looks like at the beginning of the thread.

And the graininess doesn't exist without the filter replacement?

This is a bit weird, because it makes a lot of sense that exchanging the scaling filter improves aliasing artifacts at hard edges, but I'm not sure whether it can cause "graininess" even though it's clearly a superior filter implementation.

 

For the next beta, I'll add an option to the parameters for enabling low-power mode.

The scaling issue is something I'll report to Intel. Could you help me prepare for this by running these 6 command lines?

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,vpp_qsv=width=720:height=404:format=nv12,hwdownload,format=nv12,format=argb" -y out_vpp.png 

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,vpp_qsv=width=720:height=404:format=nv12:scale_mode=1,hwdownload,format=nv12,format=argb" -y out_vpp1.png

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,vpp_qsv=width=720:height=404:format=nv12:scale_mode=2,hwdownload,format=nv12,format=argb" -y out_vpp2.png

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,scale_qsv=w=720:h=404:format=nv12,hwdownload,format=nv12,format=argb" -y out_scale.png

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,scale_qsv=w=720:h=404:format=nv12:mode=1,hwdownload,format=nv12,format=argb" -y out_scale1.png

ffmpeg.exe -init_hw_device "qsv=qd0:hw,child_device=0,qsv_use_dx11=1" -i testimg.png -filter_hw_device qd0 -filter_complex "format=yuv420p10le,hwupload=extra_hw_frames=1,scale_qsv=w=720:h=404:format=nv12:mode=2,hwdownload,format=nv12,format=argb" -y out_scale2.png

You can use the ffmpeg I sent you. You'll need to following test image in the current folder:

testimg.zip

It will create 6 images which I'd need for forwarding the issue to Intel.
Thanks, sw

Link to comment
Share on other sites

10 minutes ago, softworkz said:
1 hour ago, SoGX said:

If I delete the options again, the picture looks like at the beginning of the thread.

And the graininess doesn't exist without the filter replacement?

So as far as I can tell it's not that coarse-grained without this option. Although it's hard to judge with the low resolution at which it's transcoded.

Executed the commands, in my opinion the results don't look so good, but you can probably judge that better 😉

out_scale.png

out_scale1.png

out_scale2.png

out_vpp.png

out_vpp1.png

out_vpp2.png

Link to comment
Share on other sites

Oh, could you please zip and attach them as zip?

The forum cannot be trusted (it might "optimize" images)

Link to comment
Share on other sites

This is interesting. All your results are identical - except that those from scale_qsv indicate a non-uniform sample aspect ratio.
So it seems that it's not the scaling at all.

(my results are of better quality: scale_results.zip - it doesn't matter whether scale_qsv or vpp, only the mode makes a small difference)

 

Let's try something else:

Find: width=720:height=300
Replace: width=640:height=268

This will scale down without affecting aspect ratio

Link to comment
Share on other sites

The video looks just as bad as before. Should I create the test images again and adjust the aspect ratio in the commands?

 

Link to comment
Share on other sites

No, but let's try whether scale_qsv would still come to the rescue in this case:

Find: vpp_qsv@f1=width=720:height=300
Replace: scale_qsv@f1=width=640:height=268

 

The test in the other direction (does an uneven SAR value cause better edge encoding?)

Find: [f1_out0]
Replace: setsar=sar=300/302[f1_out0]

 

PS: It's a very mysterious case...

Link to comment
Share on other sites

5 minutes ago, softworkz said:

Find: vpp_qsv@f1=width=720:height=300
Replace: scale_qsv@f1=width=640:height=268

Better, but grainy

 

6 minutes ago, softworkz said:

Find: [f1_out0]
Replace: setsar=sar=300/302[f1_out0]

Better than the previous setting, not quite as grainy, but still far from perfect.

What I don't understand is that it works fine with jellyfin, where ffmpeg is also used.

Link to comment
Share on other sites

1 minute ago, SoGX said:

Better than the previous setting, not quite as grainy, but still far from perfect.

OK, that's enough evidence: the H264 encoder has some odd behavior depending on the sample aspect ratio.

2 minutes ago, SoGX said:

What I don't understand is that it works fine with jellyfin, where ffmpeg is also used.

Let's create a log from jf running the same file with the same setting, then we can reduce that down to identify the difference.

Link to comment
Share on other sites

Just now, softworkz said:

with the same setting

I mean transcoding quality, so we get the same resolution - not the diagnostic replacement

Link to comment
Share on other sites

Here is the transcode log from jf with the same settings (mostly default) with hw encoding enabled.

Transcoded, as with emby, to 5MBit 1080p

transcode_log_jf.txt

Link to comment
Share on other sites

But is downscaling necessary at all? On average, I have a stream of 5 MBit. That is basically what I wanted to achieve. And if it works in 1080p, all the better.

Or am I making a mistake here?

Link to comment
Share on other sites

We have just different heuristics in determining the best scale depending on your quality setting..

Try to bump up the quality in Emby 1 level.

Link to comment
Share on other sites

3 minutes ago, softworkz said:

We have just different heuristics in determining the best scale depending on your quality setting..

So far, it has always worked out great and I have been satisfied. Unfortunately, it doesn't work quite as well with this format.
But I am confident that a solution will be found sooner or later 😉

Link to comment
Share on other sites

This is not an easy subject.

At this point it might look as we just would need to change to a larger scale (where the issue is less visible).

But that's just one part - and that is very specific to your Coffee Lake system!

Because we must not forget that - even when scaling down to 720x300 - we get decent output:

  • with software encoding
  • with the same QSV encoding on other systems

Our tests have shown that on your system:

  • Scaling is of inferior quality
    and switching quality modes has no effect
  • Encoding produces inferior quality results

and that is surely not normal for a CPU like yours.

I'll wrap this up and report it to Intel. Maybe we'll need to do a few more tests, in case you'd be ready to do.

Link to comment
Share on other sites

Let's collect some more information:

  • What's the exact CPU model?
  • Which OS version?
  • Which drivers do you have installed?
    • And which was the previous version?
  • Do you have a monitor connected to the onboard graphics?

 

For the driver versions, open Device Manager, navigate to graphics, right-click => Properties, click Update Driver, choose Browse my computer, choose Let me pick from a list.
Make a screenshot like this:

image.png.505ed43b8e4cbfb7d81e250aa3988f26.png

Thanks!

Link to comment
Share on other sites

Happy2Play

@softworkz In the end this is due to Emby's algorithm/scaling for bitrate limit for 5Mbps and software vs hardware encoding differences?

How would no scaling happen in this scenario?  Would applying client quality differently honor resolution but keep bitrate limit?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...