Tone-mapping in transcoding HDR for playback on SDR screens??

[Carlo 4330]

46 minutes ago, Mkilbride said:

For example - I have a 4K HDR Remux of a movie. On my phone, it defaults to "Direct Play".  My phone however - does not support HDR. So I was getting no tonemapping. Forcing it to transcode causes it to be tonemapped and displays as expected.

Not yet.  Right now tone mapping control is only on the server and the clients are "dumb".  This will be changed of course so that clients can properly know to direct play HDR content or have the server tone map it to SDR.

So for now you would need to change the bitrate or similar on the client to force it to tone map.

[softworkz 3341]

7 hours ago, Mkilbride said:

So I just downloaded the Server Beta and tried this out. I am wondering if there's an option for devices that support native playback without trasncoding, but don't have HDR?

 

For example - I have a 4K HDR Remux of a movie. On my phone, it defaults to "Direct Play".  My phone however - does not support HDR. So I was getting no tonemapping. Forcing it to transcode causes it to be tonemapped and displays as expected.

Yes, that's not addressed yet. But we've just added tone mapping at all and we need to have on thing first before doing the other. I think, further development will depend on user feedback (hint  )

[Stahlreck 1]

8 hours ago, cayars said:

and the clients are "dumb"

Depends on the client. The apps themselves may be "dumb" but Windows for example has no problem at all tone-mapping HDR to SDR with Emby Theatre direct playing. Tested this with both the desktop and Store version. I just assume none of the apps really do anything and this is Windows doing it right? Never really looked into it but the native Movies & TV app for example has no issues with tone mapping either while some other custom media players do (old PowerDVD version I still have from the time I watched some Blu-Ray on my PC ^^).

Edited April 11, 2021 by Stahlreck

[softworkz 3341]

6 minutes ago, Stahlreck said:

Depends on the client. The apps themselves may be "dumb" but Windows for example has no problem at all tone-mapping HDR to SDR with Emby Theatre direct playing. Tested this with both the desktop and Store version. I just assume none of the apps really do anything and this is Windows doing it right? Never really looked into it but the native Movies & TV app for example has no issues with tone mapping either while some other custom media players do (old PowerDVD version I still have from the time I watched some Blu-Ray on my PC ^^).

The clients are not _that_ dumb actually. It will be up to the clients to tell the server whether they can deal with HDR content or whether they want the server to do the tone mapping in case that there's no other reason for the server to transcode. (when the server transcodes, it will always apply tone mapping when it is configured to do so, because transcoding means converting to H.264 where HDR isn't possible).

[Carlo 4330]

Hehe, I only meant "dumb" from the standpoint of HDR talking to the server.

This is new to the server beta so the clients need a small adjustment to inform the server if it can play back HDR content or not.

[Stahlreck 1]

I see. Though I guess that is dependent on the OS? I'm not on the beta right now and I assume the current apps don't really do anything about HDR and just ignore it right? And the OS either can deal with it or not, which will be fixed later for any OS that can't because the server would just tome-map upfront if the Emby app would report that the client has no HDR screen (or however that solution might look). I'm honestly just curious how it works right now on Windows.  

[softworkz 3341]

It depends on many different things (general overview, not specific to a certain OS or client):

• The OS and its video processing components
  • Whether it supports 10 bit HDR displays
    => no tone mapping required when an HDR display is connected
  • Whether it provides tone mapping capabilities as part of the video stack
    Video stack means for example: DirectShow or MediaFoundation on Windows or MediaCodec on Android
    • Could be sw implementation
    • Could be hw implemenation 
      Requires graphics drivers providing that capability
• The graphics hardware and drivers
  • Whether hw supports connecting HDR displays
  • Whether it provides hw tone mapping capability to the OSs video stack
• The connected display
  • Whether it supports HDR display
• The player implementation used by the Emby Client
  • How it interoperates with the OSs video stack
    • Whether it is capable to provide HDR video to the stack in case all other components (up to the connected display) support it
    • Whether it is capable to detect the above conditions
    • Otherwise, whether it is capable to leverage any OS-provided tone mapping capabilites
  • Otherwise, whether it includes its tone mapping capabilities
    • HW tone mapping: by accessing hw device capabilities directly (instead of the OS's video stack)
    • SW tone mapping: via custom implementation
• The Emby Client
  • How it interoperates with the embedded player implementation to make use of the available capabilities

That's probably not quite complete, but you can see that it's not as easy as one might think.

[niallobr 9]

Quick question guys. Generally speaking if a 4K HDR movie is transcoded (with vaapi) into a 1080p 4 Mbps stream for a client, should the client expect a big loss in picture quality? I can see the stream is hw accelerated, content is tone mapped but things like text and edges appear pretty jagged on screen. If a client transcodes non HDR content it seems to generally have much better overall picture quality.

[Mkilbride 76]

So this doesn't work in browser, does it?

 

It appears to tonemap if I transcode to other devices, but inside my browser it does not tonemap. 

[Carlo 4330]

1 hour ago, niallobr said:

Quick question guys. Generally speaking if a 4K HDR movie is transcoded (with vaapi) into a 1080p 4 Mbps stream for a client, should the client expect a big loss in picture quality? I can see the stream is hw accelerated, content is tone mapped but things like text and edges appear pretty jagged on screen. If a client transcodes non HDR content it seems to generally have much better overall picture quality.

Really hard to say and it can depend a lot on the original quality and how it's filmed and produced.  For example a "grainy" film will require a lot more bits to render to keep the grain look and 4Mb is pretty low for that.  I'd imagine a pure 1080-4Mb version would likely look better if mastered that way. A "clean" digital master tone mapped to 1080-4Mb would likely look better than the "grainy" version as bits are used for that grain look and feel.

If you were to tone map down to 1080-8Mb I bet the quality would look a lot better as you have a lot more bits available to reproduce the original.

I hope that makes sense. In general the more bits you can allocate with transcoding and tone mapping the better the potential result will look.

[Carlo 4330]

1 hour ago, Mkilbride said:

So this doesn't work in browser, does it?

 

It appears to tonemap if I transcode to other devices, but inside my browser it does not tonemap. 

Tone mapping right now is server controlled so if the source is HDR and it needs to transcode it should tone map to SDR for the browser.
Do you have a situation where this isn't working in a browser?

[niallobr 9]

5 hours ago, cayars said:

Really hard to say and it can depend a lot on the original quality and how it's filmed and produced.  For example a "grainy" film will require a lot more bits to render to keep the grain look and 4Mb is pretty low for that.  I'd imagine a pure 1080-4Mb version would likely look better if mastered that way. A "clean" digital master tone mapped to 1080-4Mb would likely look better than the "grainy" version as bits are used for that grain look and feel.

If you were to tone map down to 1080-8Mb I bet the quality would look a lot better as you have a lot more bits available to reproduce the original.

I hope that makes sense. In general the more bits you can allocate with transcoding and tone mapping the better the potential result will look.

Thanks for the detailed reply as always

 

Here’s some still images showing the gap in picture quality between 4K HDR and a 1080p transcode

 

4K HDR10 h265 @ 1080p 4 Mbps (Tone mapping ON)

4K HDR10 h265 @ 1080p 4 Mbps (Tone mapping OFF)

1080p h265 @ 1080p 4 Mbps

 

All of these are using vaapi hw acceleration, client is Emby for iOS. I know its not super scientific without posting source bitrates or logs… but the 1080 version does look much better at a glance? Both look good without transcoding. I feel like its something with the way 4K HDR transcodes are converted when hw acceleration is turned on? I’ll see if i can try it again with software transcoding to confirm.

[Carlo 4330]

3,1,2 in that order to my eye

[PontusN 12]

8 hours ago, niallobr said:

Thanks for the detailed reply as always

 

Here’s some still images showing the gap in picture quality between 4K HDR and a 1080p transcode

 

4K HDR10 h265 @ 1080p 4 Mbps (Tone mapping ON)

4K HDR10 h265 @ 1080p 4 Mbps (Tone mapping OFF)

1080p h265 @ 1080p 4 Mbps

 

All of these are using vaapi hw acceleration, client is Emby for iOS. I know its not super scientific without posting source bitrates or logs… but the 1080 version does look much better at a glance? Both look good without transcoding. I feel like its something with the way 4K HDR transcodes are converted when hw acceleration is turned on? I’ll see if i can try it again with software transcoding to confirm.

I had a similar problem with Tonemapping being very dark with VAAPI. See this thread for my solution:

 

[niallobr 9]

Thanks for the tip! Tone mapping definitely looks brighter when QuickSync is used… I still cant figure out why the 4k screens look so pixelated/blurry when compared to 1080. Look at the difference on the text, its pixelated on the 4k screenshot but looks pretty sharp at 1080p?

Edited April 24, 2021 by niallobr

[niallobr 9]

Update: Pixelation disappeared when I increased the transcode bitrate to > 5 Mbps. For example, if I choose 6 Mbps there is a big jump in quality and the pixelated artefacts are suddenly gone. Seems like an issue with Emby/ffmpeg h265->h264 transcode settings for 4K videos. Surely there shouldn't be a massive jump in picture quality between 5 Mbps and 6 Mbps? Maybe it will be fixed when we can do h265->h265 transcodes on the fly? In any case it's a bit off topic for this discussion but thought it might be useful to leave the note here.

Edited April 28, 2021 by niallobr

[Carlo 4330]

It's not so much the setting but what ffmpeg chooses to use in order to fit that setting.  The higher the bitrate it's "allowed" to achieve the more it can do compression wise.

[softworkz 3341]

8 hours ago, niallobr said:

Update: Pixelation disappeared when I increased the transcode bitrate to > 5 Mbps. For example, if I choose 6 Mbps there is a big jump in quality and the pixelated artefacts are suddenly gone. Seems like an issue with Emby/ffmpeg h265->h264 transcode settings for 4K videos. Surely there shouldn't be a massive jump in picture quality between 5 Mbps and 6 Mbps? Maybe it will be fixed when we can do h265->h265 transcodes on the fly? In any case it's a bit off topic for this discussion but thought it might be useful to leave the note here.

You should open up the Stats-for-Nerds view and take a look whether the resolution might have changed between 5 and 6. The resolution+Bitrate values that you see in the clients do not always give you exactly the resolution of the item you choose.

[vdatanet 1549]

Here I show you a comparison of the different tone mapping options. I find native VAAPI too dark and VAAPI Extra-T OpenCL Hable too light. QuickSync does not work for me. 

Native VAAPI (Too Dark):

 

VAAPI Extra-T OpenCL Hable (Too Light):

Intel QuickSync (Does not work):

This is the result that I would like:

 

ffmpeg-transcode-3556182a-b25e-4236-88f9-5ebb2259e5bd_1.txt ffmpeg-transcode-f1f2b3b7-9200-410e-933f-2573f7ee974e_1.txt ffmpeg-transcode-d7a12dfa-f18d-433d-b7e6-e2082a3fc03d_1.txt

Edited April 29, 2021 by vdatanet

[Carlo 4330]

That is quite a difference.  What happens if you use Reinhard or Mobius in Emby?

[vdatanet 1549]

1 minute ago, cayars said:

That is quite a difference.  What happens if you use Reinhard or Mobius in Emby?

They are very similar.

[Carlo 4330]

Do these look the same in every client?

[vdatanet 1549]

24 minutes ago, cayars said:

Do these look the same in every client?

Yes, the same in every client.

[Carlo 4330]

We'll need to see what @softworkz has to say about this.

[sooty234 266]

5 hours ago, vdatanet said:

Here I show you a comparison of the different tone mapping options. I find native VAAPI too dark and VAAPI Extra-T OpenCL Hable too light. QuickSync does not work for me. 

Native VAAPI (Too Dark):

 

VAAPI Extra-T OpenCL Hable (Too Light):

Intel QuickSync (Does not work):

This is the result that I would like:

 

ffmpeg-transcode-3556182a-b25e-4236-88f9-5ebb2259e5bd_1.txt 61.71 kB · 3 downloads ffmpeg-transcode-f1f2b3b7-9200-410e-933f-2573f7ee974e_1.txt 93.73 kB · 2 downloads ffmpeg-transcode-d7a12dfa-f18d-433d-b7e6-e2082a3fc03d_1.txt 60.98 kB · 2 downloads

That second picture looks like it isn't tone mapping. Reinhard at 0.6 would be what I would recommend.

I can't test this with hardware acceleration as my beta is running on an i7 6700k (Windows 10l, but software tone mapping is very far away from being useable. Aside from the color issues, it doesn't obey resolution settings at all. It always transcodes down to 720x404 or less.