Tone-mapping in transcoding HDR for playback on SDR screens??

[griffindodd 18]

Loving EMBY!!!!!!!!!

 

However, I have an issue with HDR files.

 

I'd like to make my growing 4K HDR library available to my family (outside of my network) to stream, but they only have regular SDR screens so playing back HDR content on their screens looks washed out and hue shifted colors due to incorrect tone mapping. 

 

Does/or will Emby adjust tone-mapping during transcode from HDR to SDR (like MadVR does) when it sees that the display device is an SDR screen?

 

Many thanks

Edited February 9, 2018 by griffindodd

[Guest asrequested]

Which clients, is it being transcoded?

Edited February 9, 2018 by Doofus

[Luke 38803]

Yea it really depends on whether transcoding is happening or not. If it's direct playing then it's subject to how the client-side video player handles it.

[griffindodd 18]

This is strictly for transcoded content. Everyone that uses my server streams and transcodes over the web, some on Roku, some on Apple TV.

 

Ideally the Emby client would know if it is attached to an HDR capable display, and if it isn't, transcodes and tone-maps the content down to SDR, much in the same way it transcodes an audio stream that isn't supported by the output device.

Edited February 9, 2018 by griffindodd

[Luke 38803]

We currently don't factor that into our transcoding decisions (although we might in the future). but for remote users there will be a good chance it is already transcoding anyway and then at that point it will be converted to sdr.

[griffindodd 18]

We currently don't factor that into our transcoding decisions (although we might in the future). but for remote users there will be a good chance it is already transcoding anyway and then at that point it will be converted to sdr.

 

Are you saying that transcoded HDR is already being converted to a correct SDR tone-map? 

Edited February 9, 2018 by griffindodd

[Guest asrequested]

It isn't purely HDR. There are multiple factors. If there isn't enough bandwidth, it will be transcoded. If they are watching on an HD display and the client can't downscale, it will be transcoded etc. And when transcoded, it will be converted to SDR. The HDR metadata won't be passed through.

[Guest asrequested]

But if the stream is played directly, the client will be responsible for how it's handled.

[griffindodd 18]

It isn't purely HDR. There are multiple factors. If there isn't enough bandwidth, it will be transcoded. If they are watching on an HD display and the client can't downscale, it will be transcoded etc. And when transcoded, it will be converted to SDR. The HDR metadata won't be passed through.

 

Yes I know how and why transcoding takes place. Simply stripping HDR metadata from a stream does not make it SDR. You have to tone-map the content so that it looks correct on a non-HDR screen.

 

I'm definitely not seeing that when I test transcoding a 4k HDR (HEVC BT2020 10bit) file to the Emby web player on a standard computer monitor for example. The image is washed out with shifted colors which shows that while the stream has of course been transcoded the video has not been accurately converted to SDR brightness range and colors.

Edited February 9, 2018 by griffindodd

[Guest asrequested]

That's curious. You may need to post logs. If I watch one of my HDR movies on my phone, the coloring appears, correct. I haven't tested in a browser. Does appear correctly if you force transcoding in one of the apps that you use?

[Guest asrequested]

I'll have to test it on my big screen. Maybe my phone screen is too small for me to see, accurately.

[griffindodd 18]

Below is a screen grab from the official Baby Driver trailer on Youtube.

 

Below is the same scene grabbed from a transcoded stream (1080p 4mbit) from my 2160p 10bit BT2020 HEVC file of the movie. You can see the contrast and color differences very clearly when you see the two side by side like this. This is a very typical example of HDR content being displayed on an SDR screen without proper tone-mapping.

 

At home I have a few HDR TVs but in my theater room I still run a 1080p SDR projector connected via HDMI to my media server where you see the exact same problem if you try and view a local HDR file (no streaming or transcoding).

 

The only software I have seen so far that does true real-time tone-mapping of HDR to SDR content is the MadVR plugin combined with Kodi - this is what I use to watch HDR content on my SDR projector and it does an excellent job. But of course this isn't any use to anyone else trying to watch my HDR files via my Emby server if they don't have a HDR capable display.

Edited February 9, 2018 by griffindodd

[Guest asrequested]

That does look like it's not converted, correctly.

 

For your projector, you should probably use Theater desktop. That uses PQ, so the colors are just the same as HDR.

 

The transcoding is through ffmpeg. Maybe the present build isn't converting, correctly?

[griffindodd 18]

That does look like it's not converted, correctly.

 

For your projector, you should probably use Theater desktop. That uses PQ, so the colors are just the same as HDR.

 

The transcoding is through ffmpeg. Maybe the present build isn't converting, correctly?

 

Like I said I use Kodi with MadVR plugin for the projector and it does an excellent job of the conversion of locally played files.

 

The issue I'm bringing up here is the ability of Emby to do the same thing when it sees an SDR screen attached to any Emby streaming client.

 

After all Emby already does similar types of checks for supported audio formats and video codecs, transcoding if the device cannot support native playback of the file. Now we need a similar check for HDR or SDR displays. HDR content is becoming more and more prevalent but SDR screens are going to be lurking around people's homes for years to come.

 

Here is a thread of discussion around HDR-SDR conversion and tone-mapping using FFMPEG...

 

https://forum.doom9.org/showthread.php?p=1828291

Edited February 9, 2018 by griffindodd

[griffindodd 18]

Maybe this information is also of help from a technical standpoint...

 

madVR is capable of tone mapping, gamut mapping and gamma transfer conversion so that any display can show HDR10 content using the limitations of its available color gamut and peak luminance. The algorithms used by madVR should be of a higher quality than those used by most UHD displays.
madVR offers five methods for dealing with HDR metadata:

passthrough HDR content to the display: The display receives the regular HDR content untouched. HDR passthrough should only be used for displays which natively support HDR playback. Sending HDR metadata to the display will be implemented in some future version. For now, you have to manually enable HDR mode in the display menu.

convert HDR content to SDR by using pixel shader math: HDR is converted to SDR. The display receives SDR content.

convert HDR content to SDR by using an external 3DLUT: HDR content is converted to SDR. The display receives SDR content. If you supply multiple 3DLUT files, the one which best matches the source gamut will be used. The 3DLUT receives untouched R'G'B' HDR (PQ) data, applies tone & gamut mapping, then outputs R'G'B' data in the display's native gamut and transfer function.

process HDR content by using pixel shader math: The display receives HDR content, but the HDR source is downconverted to the target specs. Sending HDR metadata to the display will be implemented in some future version. For now, you have to manually enable HDR mode in the display menu.

process HDR content by using an external 3DLUT: The display receives HDR content, but the 3DLUT downconverts the HDR source to some extent. The 3DLUT input/output is R'G'B' HDR (PQ). The 3DLUT applies some tone and/or gamut mapping. Sending HDR metadata to the display will be implemented in some future version. For now, you have to manually enable HDR mode in the display menu.

this display's peak nits
The display peak luminance specifies maximum display brightness. This defines the upper range of the tone mapping curve or the point where values are clipped. This applies when doing any HDR to SDR conversion or when downconverting an HDR source before passthrough. There is no such thing as a correct setting, so experiment with this value. A display configured to Rec.709 (BT.709) should start with a value of 265 nits even if it is calibrated to 100 nits. Higher peak luminance values will progressively darken the image.

[Guest asrequested]

I understand. I was suggesting that the build that is presently used by the server, may not be converting, correctly.

 

I wonder if a perceptual quantizer can be used during transcode, to solve this?

 

@@Waldonnis

Edited February 9, 2018 by Doofus

[Guest asrequested]

I just tried a different movie on my phone. Android mobile app. Yeah, it's pretty horrible.

[griffindodd 18]

I just tried a different movie on my phone. Android mobile app. Yeah, it's pretty horrible.

 

Yep it's a very real thing and something that all of the media playback community is having to deal with, SDR screens aren't going away any time soon, and having to keep a HDR and SDR version of every show/movie isn't a good alternative.

[Jdiesel 1240]

Sounds like this ffmpeg string is what is needed when converting from HDR to SDR

-vf zscale=transfer=linear,tonemap=clip,zscale=transfer=bt709,format=yuv420p

[Guest asrequested]

That looks about right. But how would it discriminate between HDR and SDR?

[griffindodd 18]

That looks about right. But how would it discriminate between HDR and SDR?

 

Yep you need to check for BT2020 in the media, and also check for HDR support on the display device, then make the decision to tone-map or not in transcode.

[Jdiesel 1240]

There are multiple ways to do it but it seems like clipping the HDR content out results in accurate SDR colors. I'm sure there are better and more complex ways of doing it through.

[Guest asrequested]

Yep you need to check for BT2020 in the media, and also check for HDR support on the display device, then make the decision to tone-map or not in transcode.

I think the end point would make everything SDR and not worry if the display can support HDR. Using PQ would make this, moot. Like mpv, does.

[griffindodd 18]

I think the end point would make everything SDR and not worry if the display can support HDR. Using PQ would make this, moot. Like mpv, does.

 

You don't want it to convert to SDR if the display device is HDR capable, even in transcode.

[Jdiesel 1240]

I think the end point would make everything SDR and not worry if the display can support HDR. Using PQ would make this, moot. Like mpv, does.

 

This is needed for transcoding though. Direct playing to Kodi or ET this isn't really an issue. Playing HDR material in any situation where transcoding it is required. This string would only be applied when transcoding HDR material.