Tonemapping Extra-T results in superior tonemapping but green line

[MSI2017 48]

Hi all,

So I've never really used tonemapping because it just looked kinda bad to me. I've now used the diagnostic plugin to change from hable to mobius and see what works best. To my surprise these settings did not fix the issues I was having, but changing from super-t to extra-t did. Colours and even the brightness curve were spot-on. Unfortunately I have this green line under y content on every device and/or app.

Anything I can do?

 

Very interestingly, when I let it transcode to a stupidly low resolution it doesn't appear

 

Edited January 7, 2023 by MSI2017
typo

[MSI2017 48]

@softworkzI belive you were most involved with the tonemapping process of Emby right? I usually try to figure stuff out myself before posting but I could not really find out the difference between Super and Extra T.

[softworkz 5067]

7 hours ago, MSI2017 said:

@softworkzI belive you were most involved with the tonemapping process of Emby right? I usually try to figure stuff out myself before posting but I could not really find out the difference between Super and Extra T.

I have done two different implementations of OpenCL tone mapping for Intel. One of them is doing all of the steps inside the OpenCL filter and in the other case, YUV/RGB conversion is done by Intel VPP before and after the OpenCL filter. 

The results are always the same for both ways. The only difference is performance. With some hardware one way is faster and with other hardware the other one. Also there were cases where Extra-T (non-default) wasn't working at all. I wasn't able to derive any reliable rules to auto-choose between the two, that's why it ended up as an option.

[MSI2017 48]

On 1/7/2023 at 9:21 PM, softworkz said:

I have done two different implementations of OpenCL tone mapping for Intel. One of them is doing all of the steps inside the OpenCL filter and in the other case, YUV/RGB conversion is done by Intel VPP before and after the OpenCL filter. 

The results are always the same for both ways. The only difference is performance. With some hardware one way is faster and with other hardware the other one. Also there were cases where Extra-T (non-default) wasn't working at all. I wasn't able to derive any reliable rules to auto-choose between the two, that's why it ended up as an option.

Interesting, for me it makes a huge difference. I think is Super-T is chosen YUV/RGB conversion is not being handled (properly) since I get really washed out colours. 

 

Any thoughts about the green line? Other than that it is pretty much perfect

Edited January 14, 2023 by MSI2017
typo's

[softworkz 5067]

2 minutes ago, MSI2017 said:

Interesting, for me it makes a huge differnece. I think is Super-T is chosen YUV/RGB conversion is not being handles (properly) since I get really washed out colours. 

Windows or Linux?

2 minutes ago, MSI2017 said:

Any thoughts about the green line?

This is about alignment requirements (w/h need to be multiples of 8,16,32, etc - depending on the case).

Could you please post ffmpeg logs showing both Super-T and Extra-T?

[MSI2017 48]

Just now, softworkz said:

Windows or Linux?

This is about alignment requirements (w/h need to be multiples of 8,16,32, etc - depending on the case).

Could you please post ffmpeg logs showing both Super-T and Extra-T?

Windows, yes will tho. Need to change any log settings?

[MSI2017 48]

On 1/7/2023 at 9:28 PM, softworkz said:

Windows or Linux?

This is about alignment requirements (w/h need to be multiples of 8,16,32, etc - depending on the case).

Could you please post ffmpeg logs showing both Super-T and Extra-T?

I have sent them in DM, when you have the time

[MSI2017 48]

I'll put the logs here just in case someone else might have an idea.

https://katb.in/cotogukatuh This is super-T

https://katb.in/jisewerarur This is Extra-T

 

Something else I noticed, but not tonemapping related is that often the playback gets stuck whenever you skip forward either too often (too many forward clicks) or just click th progressbar too far. I've attached those logs to this message. Maybe @cayars has an idea? I know before you figured out a playback issue for me in a few minutes when I couldn't find it at all.

 

PlaybackStopHDR.rtf

Edited January 14, 2023 by MSI2017
Wrong log

[visproduction 315]

The end encode height needs to be a multiple of 16 exactly.  If you set that to the correct height, that should fix it.

https://community.intel.com/t5/Media-Intel-oneAPI-Video/ffmpeg-H264-qsv-encoding-generates-horizontal-green-bar-at-the/td-p/1136138
&
https://stackoverflow.com/questions/60778078/ffmpeg-h264-qsv-green-line-at-bottom-of-video

[MSI2017 48]

9 minutes ago, visproduction said:

The end encode height needs to be a multiple of 16 exactly.  If you set that to the correct height, that should fix it.

https://community.intel.com/t5/Media-Intel-oneAPI-Video/ffmpeg-H264-qsv-encoding-generates-horizontal-green-bar-at-the/td-p/1136138
&
https://stackoverflow.com/questions/60778078/ffmpeg-h264-qsv-green-line-at-bottom-of-video

Any way to fix? Because otherwise it would mean that all 1920x1080 files and 3840x2160 files would have the green line

 

Plus those links seem to be about QSV, Here it only happens when selecting a specific way of tonemapping

Edited January 14, 2023 by MSI2017

[MSI2017 48]

@softworkz found this on reddit, but a more recent post on ffmpeg saying that there is a new way. Could this be the cause?

 

[Luke 42079]

On 1/11/2023 at 5:54 PM, MSI2017 said:

I have sent them in DM, when you have the time

Can you please add me to that? Thanks.

[MSI2017 48]

1 minute ago, Luke said:

Can you please add me to that? Thanks.

Done, thank you for the help!

[MSI2017 48]

@Luke @softworkz When reading through the 4.8 beta releases I saw the QSV green line fix in there. I pressume this also applies to extra-t but will make sure to test when it releases. Therefore I think you can let this go. Although it is interesting that on super-t tonemapping looks really bad.

 

Also congrats on the SDK, just saw the video!

[visproduction 315]

On 1/14/2023 at 10:13 AM, MSI2017 said:

Any way to fix? Because otherwise it would mean that all 1920x1080 files and 3840x2160 files would have the green line

 

Well, you could try setting the encoder to add top and bottom 4px black bars to make it 1920 x 1088, which is divisble by 16.  The 4K 2160 is already fine and dvisible by 16.  Adding top and bottom black lines is easy enough, if you set the encoding command line, per media task, but might be more tricky to make it automatic.  Anytime a non divisible by 16 height shows up, add top and bottom black to the encoding target height so that the height is exactly divisible by 16.  You could prep the encode command line with some .js math code.  Does ffmpeg have an easier way to do this?   Isn't that code all under the hood, so to speak, when media is transcoded with Emby?  Then there is the problem of playing back 1920 x 1088 on a 16:9 TV.  What does the TV actually do?  Does it shrink the width, as well, to fit on the monitor?  If so, then it has to blur the pixels from 1920 to around 1912 width which will lose some resolution.

It seems like a better answer is to just use 1280 x 720 (720P) content, which doesn't have the divisible by 16 problem.

[MSI2017 48]

1 hour ago, visproduction said:

Well, you could try setting the encoder to add top and bottom 4px black bars to make it 1920 x 1088, which is divisble by 16.  The 4K 2160 is already fine and dvisible by 16.  Adding top and bottom black lines is easy enough, if you set the encoding command line, per media task, but might be more tricky to make it automatic.  Anytime a non divisible by 16 height shows up, add top and bottom black to the encoding target height so that the height is exactly divisible by 16.  You could prep the encode command line with some .js math code.  Does ffmpeg have an easier way to do this?   Isn't that code all under the hood, so to speak, when media is transcoded with Emby?  Then there is the problem of playing back 1920 x 1088 on a 16:9 TV.  What does the TV actually do?  Does it shrink the width, as well, to fit on the monitor?  If so, then it has to blur the pixels from 1920 to around 1912 width which will lose some resolution.

It seems like a better answer is to just use 1280 x 720 (720P) content, which doesn't have the divisible by 16 problem.

using 720p are a few backwards steps too far. But I believe this has already been patched in 4.8

[softworkz 5067]

On 1/14/2023 at 7:35 PM, MSI2017 said:

@softworkz found this on reddit, but a more recent post on ffmpeg saying that there is a new way. Could this be the cause?

 

 

That merge of  scale_qsv into vpp_qsv has never happened because I had asked Intel not to do it.

There's an important difference for which it is sometimes preferable to use scale_qsv over vpp_qsv.

I will look into the green-bar issue shortly. If it is really(!) related to scale_qsv, then I know what it is as I had already fixed it in an earlier version of our ffmpeg. But it could also be for another reason.

[MSI2017 48]

6 minutes ago, softworkz said:

 

That merge of  scale_qsv into vpp_qsv has never happened because I had asked Intel not to do it.

There's an important difference for which it is sometimes preferable to use scale_qsv over vpp_qsv.

I will look into the green-bar issue shortly. If it is really(!) related to scale_qsv, then I know what it is as I had already fixed it in an earlier version of our ffmpeg. But it could also be for another reason.

I mentioned some further info in our DM, I believe it might actually already be fixed in 4.8, but would have to test on release if it is als the case for extra-T.

[softworkz 5067]

8 minutes ago, MSI2017 said:

I believe it might actually already be fixed in 4.8, but would have to test on release if it is als the case for extra-T.

I'm confused. There's not 4.8 yet, so which "release" are you talking about? On which exact version do you encounter the issue?

[MSI2017 48]

Just now, softworkz said:

I'm confused. There's not 4.8 yet, so which "release" are you talking about? On which exact version do you encounter the issue?

i am on stable release, but I checked the 4.8 beta and saw the green line issue as resolved in the changelog

[softworkz 5067]

1 minute ago, MSI2017 said:

i am on stable release, but I checked the 4.8 beta and saw the green line issue as resolved in the changelog

Got it, thanks!

Yes, would be great when you could test it in the beta.

[MSI2017 48]

3 hours ago, softworkz said:

Got it, thanks!

Yes, would be great when you could test it in the beta.

I'm afraid my test-system uses nvidia for transcoding. Would have to test on main, can you easily revert back to stable or when you go beta there's no road back?

[softworkz 5067]

Are you on Windows?

[MSI2017 48]

1 minute ago, softworkz said:

Are you on Windows?

Yes

[softworkz 5067]

1 minute ago, MSI2017 said:

2 minutes ago, softworkz said:

Are you on Windows?

Yes

Then you can simply use the portable install: https://github.com/MediaBrowser/Emby.Releases/releases/download/4.8.0.21/embyserver-win-x64-4.8.0.21.7z

Unzip anywhere and you'll have a totally separate and independent Emby Server.

You just need to stop your primary server (or use different ports).