Jump to content

HEVC Profile Main 10 / QSV Hardware Transcoding / staircase effect


SoGX
Go to solution Solved by softworkz,

Recommended Posts

Hello everyone,

I currently have a problem with hardware transcoding when I use the QSV hardware transcoding.
This can be seen when transcoding HEVC videos with the profile "Main 10", i.e. 10 bits. Here, staircase effects occur during transcoding. If I use the software encoder, everything works perfectly.
Likewise, everything works well with HEVC videos with the profile "Main", i.e. 8 bit.

What could be the reason for this, or how can I solve the problem?

 

HEVC 10 Bit QSV.png

HEVC 10 Bit Software.png

Link to comment
Share on other sites

  • 2 weeks later...

Hallo zusammen,

beim Transcodieren von H265 Videos mit demrofil "Main 10" erscheinen unschöne Treppeneffekte.

Beim Software Transcoding treten diese nicht auf, ebenfalls nicht bei H265 Videos mit dem Profil "Main"

HEVC 10 Bit QSV.png

HEVC 10 Bit Software.png

Edited by SoGX
Link to comment
Share on other sites

visproduction

The Original media is 1920 x 804 which is almost exactly 2.39 aspect ratio. (2.3880597)

The output on this line of the first ffmpeg-transcode text shows 720 x 300 is a slightly different aspect ratio: 2.4.  This is often a compromise ratio used for encoding.

  21:54:34.437   Stream #0:0: Video: h264, qsv(progressive), 720x300 [SAR 1:1 DAR 12:5], q=2-31, 4616 kb/s, Level 30, 23.98 fps, 90k tbn


For this stream output playback in 720 x 300 everything will be compressed vertically by 0.5%.  This could be the problem that shows up with the moving logo at the beginning of the video.In addition reducing the pixel size to 37.5% of the original introduces additional pixelization.  It would be preferred to make the reduction size to exactly either 50% or 25% which converts 2 pixels to 1 or 4 pixels to 1.  Your size choice converts 200 pixels to 75 or 10 pixels to 3.75.  When you get into non-whole numbers, the process is to blur across pixels to solve the size change.  Your resize choice is causing the pixelization and it's more noticable with zooming logo text.

Try either forcing the output to not change the original aspect ratio at all and see if that helps.  I don't know where that option is.  I only use dedicated encoders to transcode.

Hope that helps.

Link to comment
Share on other sites

@softworkz

Hello,

Thank you very much for the explanation. Of course, that explains the poor quality. Unfortunately, I have little influence on it. The difference between the first and the second stream is simply that I have activated and deactivated the hardware transcoding.
The output format is set to 1080p 5MBit in both cases.

As far as I can tell, the problem only occurs with films with the profile "Main 10".

Link to comment
Share on other sites

With jellyfin installed on the same server it works like expected.

I attached the jellyfin transcoding log, maybe this can help.

Log.txt

Edited by SoGX
Link to comment
Share on other sites

@SoGX - this is a self-made problem.

You have configured CRF 18 for the software encoder:

image.png.eef9a23240d123a0b005740bb3d1a490.png

and you made no settings for the QSV encoder.

With default setting, the libx264 encoding quality is already a bit better than than the hw encoders, now when you tweak libx264 for better quality, the gap will become even more visible.

It's not your fault, though. In the current version, the UI doesn't give you the slightest hint that the setting you make will affect sw encoding only. It's a long-term flaw, unfortunately.

You can change QSV encoding parameters by installing the Diagnostics plugin (another point that we'll hopefully be able to improve soon).
Then, go to "Advanced Transcoding" and click on the buttons next to the QSV H.264 encoding to change QSV encoding quality.

Link to comment
Share on other sites

@softworkzThanks for your answer and for the tips

I have installed the Diagnostics Plugin and made a few adjustments in the h264 QSV settings.
Unfortunately, the quality of the video remains poor.
What surprises me is that the problem only affects 10bit h265 videos, the problem does not occur with 8bit h265 videos.
I have sent you a pm with a link to a sample video, maybe that will help a little.

 

 

Link to comment
Share on other sites

  • 2 weeks later...

Hello,

is there any news regarding this problem. Could it be reproduced?
As I see it, the problem seems to be that 10 bit videos seem to be played back in 720 x 300 resolution as soon as you transcode them with QSV (in my case 1080p with 5MBit).

Edited by SoGX
Link to comment
Share on other sites

On 7/5/2022 at 7:03 PM, visproduction said:

The Original media is 1920 x 804 which is almost exactly 2.39 aspect ratio. (2.3880597)

The output on this line of the first ffmpeg-transcode text shows 720 x 300 is a slightly different aspect ratio: 2.4.  This is often a compromise ratio used for encoding.

Well spotted. 

It's not a "compromise ratio", though, it's about math. Width and height values need to be multiples of 2.

720 / 1920 * 804 = 301.5.

Now, 302 would be closer to 301.5 than 300, but experience has shown that it is safer to round to the lower value because rounding to the higher can sometimes introduce an ugly line of undefined color at the bottom.

ffmpeg.zip

Link to comment
Share on other sites

@SoGX - I don't see that effect with the exact same command line.

Step 1

Please install the latest drivers from the Intel website.
Retry.

Step 2

Please replace the ffmpeg version with the one attached.
Retry.
(yours is located in C:\Users\Administrator.INTERN\AppData\Roaming\Emby-Server\system\)

ffmpeg.zip

Link to comment
Share on other sites

I'm not sure what it could be at this point. Where/how do you view the output? (client)

Let's try this:

  • Got to your transcoding temp (C:\Emby-Cache\transcoding-temp)
  • Delete everything
  • Start playback with hwa
  • Zip the first 10 generated segments
  • Post the zip here (or PM)

 

Link to comment
Share on other sites

On the other hand, if it's a hardware problem, why does it run flawlessly with Jellyfin on the same server? Also with hardware transcoding.
I use Firefox, Chrome and a FireTV as clients.

Transcode.zip

Link to comment
Share on other sites

Thanks. Could you please install the diagnostics plugin and make the following settings for Parameter Adjustment:

image.png.ffaf3f3af5456d7876b7637873829713.png

Link to comment
Share on other sites

Thanks, let's try some  more replacements:

Test 1

Find: -level:v:0 30
Replace: -level:v:0 51

Test 2

Find: -load_plugin:v:0 hevc_hw
Replace:            (leave empty)

Test 3

Find: vpp_qsv
Replace:  scale_qsv

Test 4

Find: vpp_qsv@f1=
Replace:  vpp_qsv@f1=scale_mode=2:

 

You don't need to show results for any of the tests. Just let me know whether any of those will make a change.

Edited by softworkz
Link to comment
Share on other sites

Test 1,2 and 4 do not seem to make much difference. In test 3, the staircase effect no longer seems to be there, but the image looks very coarse.

Link to comment
Share on other sites

Test 5

Find: vpp_qsv@f1=
Replace:  scale_qsv@f1=mode=2:

(you can also try 1 for the mode with both filters)

 

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...