Jump to content

Tone-mapping in transcoding HDR for playback on SDR screens??


griffindodd
Go to solution Solved by Luke,

Recommended Posts

1 hour ago, softworkz said:

There is no forcing of software tone mapping.

To be honest, I'm not sure whether the dropdown selection is a good choice. The original concept is what you can see in the Diagnostics Plugin. There are two checkboxes, one for enabling software tone mapping and another one for enabling hardware tone mapping:

image.png.5b12743dda69ad9b7d2d2e60232ea7e0.png

...which translates to: "use sw tone mapping when sw transcoding" and "use hw tone mapping when hw transcoding"
But those options have no influence on whether sw or hw transcoding is used.

Would the two-checkbox variant be more clear?

When "Enable hardware acceleration when available" is set to no, is there any point in showing the HW tone mapping option to users? Also, would there be any scenario where users would want software tone mapping only, even when hardware tone mapping was available to them?

Edited by kribby
Link to comment
Share on other sites

29 minutes ago, kribby said:

When "Enable hardware acceleration when available" is set to no, is there any point in showing the HW tone mapping option to users? Also, would there be any scenario where users would want software tone mapping only, even when hardware tone mapping was available to them?

As of today, enabling tone mapping with cpu only is probably only for those who want to try it out of curiosity. I'm not sure many will want to actually do that on a regular basis. Over time of course, with newer machines and improvements to the algorithms, that could change.

Link to comment
Share on other sites

29 minutes ago, kribby said:

When "Enable hardware acceleration when available" is set to no, is there any point in showing the HW tone mapping option to users? Also, would there be any scenario where users would want software tone mapping only, even when hardware tone mapping was available to them?

Kribby,

I prefer when options are explicit in what they do. What you are describing sounds like emby should be making these decisions for the user.

In an ideal world emby would install and have no options, it would do its job with the least amount of configuration as possible. To do that the software would need way more information than it has access to. In this instance it can not possibly know what is the best situation for a users given use case + hardware configuration.

with that I I appreciate the very explicit configuration attempt. The only bit that is unclear is priority order. If both are enabled I want a way to order which is used first and I would expect the other to work as a fallback.

If the other is never used as a fallback then I would expect to only be able to select one of these options at a time.

 

 

Link to comment
Share on other sites

40 minutes ago, kribby said:

When "Enable hardware acceleration when available" is set to no, is there any point in showing the HW tone mapping option to users? 

In case of the combo box selection yes!

Imagine the following situation:

  • User has hwa enabled
  • User has set tone mapping to hw
  • Now, user sets hwa to disabled
  • What should the drop down show now, when you say it should hide the hw tone mapping option?
    • An empty entry?
    • The software entry?
    • The None entry?
  • When the user saves next time, he would save that change to the server
  • Now, he re-enables hwa
  • But hw tone mapping is no longer enabled
    => which is unexpected, the user has just disabled and re-enabled hwa, but that shouldn't modify other options!

 

40 minutes ago, kribby said:

Also, would there be any scenario where users would want software tone mapping only, even when hardware tone mapping was available to them?

Yes, in many cases:

  • HW cannot do tone mapping
  • HW doesn't support a certain transcoding setup (e.g. codec, pixel format, profile, level, etc.)
  • Processing is forced to happen in software (e.g. due to subtitle burn-in)
  • A user's CPU is faster than the GPU
    (hw tone mapping is not always so much in advance as you might think, as long as you don't have an Nvidia card)

Our CPU tone mapping is pretty fast, have you even tried?

Link to comment
Share on other sites

26 minutes ago, Luke said:

As of today, enabling tone mapping with cpu only is probably only for those who want to try it out of curiosity. I'm not sure many will want to actually do that on a regular basis. 

You seem to have missed Rigg's feedback..

26 minutes ago, Luke said:

Over time of course, with newer machines and improvements to the algorithms, that could change.

The implementation is already pretty much at the edge of what's possible, using a lot of tricks. Parallelism is the key for getting even faster. Our implementation is using up to SSE2 features. A separate optimized implementation for AVX or AVX2 would be possible, but limited to newer CPU models.

Edited by softworkz
Link to comment
Share on other sites

32 minutes ago, cryzis said:

with that I I appreciate the very explicit configuration attempt. The only bit that is unclear is priority order. If both are enabled I want a way to order which is used first and I would expect the other to work as a fallback.

You actually CAN order the codecs. The topmost has the highest priority.

32 minutes ago, cryzis said:

If the other is never used as a fallback then I would expect to only be able to select one of these options at a time.

It works like this:

  • For each transcoding, Emby tries to find a matching hw codec setup
  • First, It will look for a suitable encoder
  • It starts with the first one in the list (according to the order you have conrfigured)
  • Each encoder is checked whether it's suitable by checking max resolution and framerate, supported codec profiles and levels, supported color formats and one or two other things
  • It does that until it finds one, otherwise, software encoding is used
  • If a hw encoder is found, a similar process starts for decoders, only that now it also checks whether the decoder can be combined with the encoder (same hw context)

So, your order DOES really matter. It's just that when you have a very recent CPU+iGPU and a recent discrete GPU, you might rarely see that the second in the list is chosen because both GPUs can do the same. (unless you try an 8k video for example..

Link to comment
Share on other sites

48 minutes ago, softworkz said:

In case of the combo box selection yes!

Imagine the following situation:

  • User has hwa enabled
  • User has set tone mapping to hw
  • Now, user sets hwa to disabled
  • What should the drop down show now, when you say it should hide the hw tone mapping option?
    • An empty entry?
    • The software entry?
    • The None entry?
  • When the user saves next time, he would save that change to the server
  • Now, he re-enables hwa
  • But hw tone mapping is no longer enabled
    => which is unexpected, the user has just disabled and re-enabled hwa, but that shouldn't modify other options!

 

Yes, in many cases:

  • HW cannot do tone mapping
  • HW doesn't support a certain transcoding setup (e.g. codec, pixel format, profile, level, etc.)
  • Processing is forced to happen in software (e.g. due to subtitle burn-in)
  • A user's CPU is faster than the GPU
    (hw tone mapping is not always so much in advance as you might think, as long as you don't have an Nvidia card)

Our CPU tone mapping is pretty fast, have you even tried?

I apologize for asking questions before, because it appears I still don't have a solid grasp on how all this works yet. Is it technically possible to have a hardware accelerated transcode using software tone mapping?

Link to comment
Share on other sites

2 minutes ago, kribby said:

I apologize for asking questions before, because it appears I still don't have a solid grasp on how all this works yet. Is it technically possible to have a hardware accelerated transcode using software tone mapping?

Technically, yes - but we don't do that right now. We do mixed processing pipelines, though, where one side is hw and the other sw or vice-versa, and for subtitle overlay also hw on both sides (encoding and decoding). But when one side is hw, we also try to setup hw tone mapping. 

It's a complex logic and as of now, I'm sure to say that nobody can do this better than us 🙂 

Link to comment
Share on other sites

8 minutes ago, softworkz said:

Technically, yes - but we don't do that right now. We do mixed processing pipelines, though, where one side is hw and the other sw or vice-versa, and for subtitle overlay also hw on both sides (encoding and decoding). But when one side is hw, we also try to setup hw tone mapping. 

It's a complex logic and as of now, I'm sure to say that nobody can do this better than us 🙂 

Oh, okay. That would explain why the same scenario didn't work for me the other day. If tonemap_vaapi doesn't have any option to change the algorithm, then I could see the importance of mixing HW transcoding with SW tonemapping 

Link to comment
Share on other sites

20 minutes ago, kribby said:

Oh, okay. That would explain why the same scenario didn't work for me the other day. If tonemap_vaapi doesn't have any option to change the algorithm, then I could see the importance of mixing HW transcoding with SW tonemapping 

In the next (or subsequent) beta, you'll be able to use OpenCL tone mapping with VAAPI as an alternative, and there you can change the algorithm as well.

Link to comment
Share on other sites

@softworkzFollow up question to a previous post I made. Transcoding stopped working for me with intel-VAAPI for some reason*** (Ubuntu). So I installed intels media sdk to get quicksync instead. This works, but tonemapping does not for some reason. According to the logs everything seems to work just fine (by my eyes), but the colors are still washed out. 

 

***This was the reason, it tried using HW VAAPI but fails and falls back to SW:
 

14:51:19.855   Metadata:
14:51:19.855     encoder         : Lavf58.35.100
14:51:19.855     Stream #0:0: Video: h264 (h264_vaapi) (High), vaapi_vld, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 6616 kb/s, Level 31, 23.98 fps, 90k tbn, 23.98 tbc (default)
14:51:19.855     Metadata:
14:51:19.855       encoder         : Lavc58.62.100 h264_vaapi
14:51:19.855     Stream #0:1(eng): Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp, 192 kb/s (default)
14:51:19.855     Metadata:
14:51:19.855       encoder         : Lavc58.62.100 libmp3lame
14:51:19.904 [tonemap_vaapi@f2 @ 0x15a5180] No mastering display data from input
14:51:19.904 Error while filtering: Invalid argument
14:51:19.904 Failed to inject frame into filter network: Invalid argument
14:51:19.904 Error while processing the decoded data for stream #0:0
14:51:19.914 [libmp3lame @ 0x10dbe00] 3 frames left in the queue on closing
14:51:19.950 Conversion failed!

ffmpeg-transcode-7a153df8-ba9a-4024-97de-0be6c33e8090_1.txt

Edited by PontusN
Link to comment
Share on other sites

I convert all my files using sickbeard_mp4_automator and have intel-VAAPI hardware transcoding on my server. In the two files below the tracks are very similar since they've been converted using the same process, however one file gets transcoded using tone mapping with hardware acceleration while the other gets a software transcode for the tone mapping. Any idea what would cause this would happen when the files are so similar?

hardware-transcode.jpg

software-transcode.jpg

Link to comment
Share on other sites

On 3/4/2021 at 3:41 PM, PontusN said:

@softworkzFollow up question to a previous post I made. Transcoding stopped working for me with intel-VAAPI for some reason*** (Ubuntu). So I installed intels media sdk to get quicksync instead. This works, but tonemapping does not for some reason. According to the logs everything seems to work just fine (by my eyes), but the colors are still washed out. 

 

***This was the reason, it tried using HW VAAPI but fails and falls back to SW:
 


14:51:19.855   Metadata:
14:51:19.855     encoder         : Lavf58.35.100
14:51:19.855     Stream #0:0: Video: h264 (h264_vaapi) (High), vaapi_vld, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 6616 kb/s, Level 31, 23.98 fps, 90k tbn, 23.98 tbc (default)
14:51:19.855     Metadata:
14:51:19.855       encoder         : Lavc58.62.100 h264_vaapi
14:51:19.855     Stream #0:1(eng): Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp, 192 kb/s (default)
14:51:19.855     Metadata:
14:51:19.855       encoder         : Lavc58.62.100 libmp3lame
14:51:19.904 [tonemap_vaapi@f2 @ 0x15a5180] No mastering display data from input
14:51:19.904 Error while filtering: Invalid argument
14:51:19.904 Failed to inject frame into filter network: Invalid argument
14:51:19.904 Error while processing the decoded data for stream #0:0
14:51:19.914 [libmp3lame @ 0x10dbe00] 3 frames left in the queue on closing
14:51:19.950 Conversion failed!

ffmpeg-transcode-7a153df8-ba9a-4024-97de-0be6c33e8090_1.txt 400.85 kB · 1 download

@PontusN - Tone mapping (especially with VAAPI) does not work when a video doesn't include the required metadata 

Link to comment
Share on other sites

not sure if should ask here but i am doing this for tone mapping anyway and its just related.... so i installed a gtx 1050 to use for hardware transcoding etc.. I have done the patch to make unlimited from the 2 limit.. but its still only letting me do 2 on emby and the 3rd is just software transcoding... is there something I have to do or is the patch just not working ?

Edited by vick1982
Link to comment
Share on other sites

Did you test it outside Emby to make sure you got the patch done correctly?

Link to comment
Share on other sites

Just now, cayars said:

Did you test it outside Emby to make sure you got the patch done correctly?

how do i do that ? sorry if thats a dumb question 

Link to comment
Share on other sites

Got the unlock patch working now...

Just a question... How will it know to tone map on direct play with a device that doesn't have HDR ? Or it won't tone map for direct play ?

Link to comment
Share on other sites

18 minutes ago, vick1982 said:

Or it won't tone map for direct play ?

If the item is direct playing then the only opportunity for tone-mapping will be on the end device.   The server is just feeding the file.

What we still need to build in is a way for apps to tell the server that it needs to transcode in order to do the tone-map. 

Link to comment
Share on other sites

So for right now with tone mapping "smarts" only in the server and not added to apps yet you would have to force a transcode if it was direct playing and you're device can't tone map.

Link to comment
Share on other sites

not a big deal.. but would be nice if it also showed it was tone-mapping in the dashboard / active devices screen

  • Like 1
Link to comment
Share on other sites

12 minutes ago, vick1982 said:

not a big deal.. but would be nice if it also showed it was tone-mapping in the dashboard / active devices screen

Until it will be added to the main UI:

 

  • Like 3
Link to comment
Share on other sites

41 minutes ago, softworkz said:

Until it will be added to the main UI:

thanks for the reply... that's great

Edited by softworkz
Moved reply text out of quote region
  • Like 1
Link to comment
Share on other sites

  • 4 weeks later...

So I just downloaded the Server Beta and tried this out. I am wondering if there's an option for devices that support native playback without trasncoding, but don't have HDR?

 

For example - I have a 4K HDR Remux of a movie. On my phone, it defaults to "Direct Play".  My phone however - does not support HDR. So I was getting no tonemapping. Forcing it to transcode causes it to be tonemapped and displays as expected.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...