Tonemapped playback thumbnails

[fredflix 23]

Now that Emby support tonemapping of HDR content, it would be nice if the playback thumbnails Emby generates could be tonemapped as well, because as of now, they're pretty washed out.

[Luke 37063]

Hi, try deleting your existing ones and refreshing metadata to allow the server to regenerate them. We did do a little work on this. It's not a full tone mapping but it should be a little better than before.

@softworkz

 

[fredflix 23]

1 hour ago, Luke said:

Hi, try deleting your existing ones and refreshing metadata to allow the server to regenerate them. We did do a little work on this. It's not a full tone mapping but it should be a little better than before.

@softworkz

 

How did I delete the thumbnails may I ask?
I renamed the HDR x265 movie from mymovie.mkv to mymovie.ignore , then refreshed the library.
Movie is gone from the library. Then I rename the movie to its original mymovie.mkv name, refreshed the library, and the new thumbnails generated are still the same washed out ones. I did refreshed the metadata as well.
I know these thumbnails are new ones as I could see the ffmpeg process running 100% on 1 of my CPU cores during the generation of the thumbs.

Here's the ffmpeg being used when generating the thumbs:
 

/app/emby/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:/mnt/movies/mymovie.mkv -an -sn -vf scale=w=320:h=133,format=yuv420p,eq=contrast=0.9:gamma=0.95:saturation=1.8 -vsync cfr -r 0.1 -f image2 /config/cache/temp/e10dfb065da64e8497d4c508c8393436/img_%05d.jpg

I don't really know any other way to delete the thumbnails.

Edited May 28, 2021 by fredflix
Added ffmpeg command used when thumbs are generated

[softworkz 3335]

Yes, that's the current state of what we're doing. It's better than before, yet not real tone-mapping. But: it's predictable and won't cause thumbnail images to be totally off-range as it can happen with real tone mapping.

Tone-Mapping is a brand new feature in Emby and we still need to gather more experience about how this will work out for users. Applying tone mapping to thumbnail generation has a different kind of impact than when just watching a movie:

In that case, and when it looks wrong, you can stop, make some adjustments to the tone mapping settings and continue watching.

For thumbnails, it's different: those images are made to stay and it's not too easy to have them regenerated - as you have had just found out...
So, what TM settings to use for thumbnail generation? The user's current TM settings? Well - these might have just been adjusted for displaying a certain video, but may not be suitable for as general setting for thumbnail generation. These results can be much worse than what we're currently doing.
That's why we are rather staying on the "safe side" for now, until we have figured out a solid procedure that is suitable for all.

[fredflix 23]

30 minutes ago, softworkz said:

Yes, that's the current state of what we're doing. It's better than before, yet not real tone-mapping. But: it's predictable and won't cause thumbnail images to be totally off-range as it can happen with real tone mapping.

Tone-Mapping is a brand new feature in Emby and we still need to gather more experience about how this will work out for users. Applying tone mapping to thumbnail generation has a different kind of impact than when just watching a movie:

In that case, and when it looks wrong, you can stop, make some adjustments to the tone mapping settings and continue watching.

For thumbnails, it's different: those images are made to stay and it's not too easy to have them regenerated - as you have had just found out...
So, what TM settings to use for thumbnail generation? The user's current TM settings? Well - these might have just been adjusted for displaying a certain video, but may not be suitable for as general setting for thumbnail generation. These results can be much worse than what we're currently doing.
That's why we are rather staying on the "safe side" for now, until we have figured out a solid procedure that is suitable for all.

I just thought it'd be a simple matter of "if media is HDR, generate thumbnails with tonemapping filters".
Thanks for the in-depth explanation
 

[softworkz 3335]

8 minutes ago, fredflix said:

I just thought it'd be a simple matter of "if media is HDR, generate thumbnails with tonemapping filters".
Thanks for the in-depth explanation
 

You're welcome! And you have no idea how much I wish it was like that...  

[rbjtech 4260]

I cheat and just copy the .bif file from my SDR copy ...

Even with the gamma and contrast corrections, the HDR generated thumbnails on an SDR display are very washed out - but better than they were, so small steps and all that .. 

 

[amb13 17]

For my movies which I have in both HDR and SDR I group them together. As far as I can tell both the chaptermarkes and thumbnails are from the HDR copy of the movie even when watching the 1080p SDR version. So when i scroll the timeline I see washed  out pictures for both SDR and HDR. Would it be possible to alway choose the sdr .bif file for botth SDR and HDR movie version?

[rbjtech 4260]

Emby doesn't re-read the BIF/Chapter files once it has a record of them in the dB for a particular item - so depending what was created first, that is what emby will use for both versions.  I have found the best way to do this is to simply remove the HDR version of the BIF and replace with the SDR version - keeping the HDR file naming obviously.  Then do a refresh on that item to replace the references in the database.  It's a pain I agree - but you only need to do it once.

[cwills75 13]

I've been screwing around with this, manually creating tone-mapped .bif files and over-writing the ones Emby creates with UHD content.  These are the four different options I've tried:

1:

ffmpeg -f matroska -skip_interval 10 -copyts -i file:"input.mkv" -an -sn -vf "zscale=transfer=linear,tonemap=hable,zscale=transfer=bt709" -s 320x180 -vsync cfr -r 0.1 -f image2 bif/%08d.jpg

2:

ffmpeg -f matroska -skip_interval 10 -copyts -i file:"input.mkv" -an -sn -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p" -s 320x180 -vsync cfr -r 0.1 -f image2 bif/%08d.jpg

3:

ffmpeg -f matroska -skip_interval 10 -copyts -i file:"input.mkv" -an -sn -vf "zscale=transfer=linear,tonemap=mobius,zscale=transfer=bt709" -s 320x180 -vsync cfr -r 0.1 -f image2 bif/%08d.jpg

4:

ffmpeg -f matroska -skip_interval 10 -copyts -i file:"input.mkv" -an -sn -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=mobius:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p" -s 320x180 -vsync cfr -r 0.1 -f image2 bif/%08d.jpg

Output bif folders then turned into .bif files with biftool:

biftool -t 10000 bif

To me, option #4 gives the most pleasing results.  Here is an example of the four outputs.

[cwills75 13]

And here is option 4 compared to Emby's original .bif output.  It would be nice in the future for Emby to automatically tone-map these properly when it detect UHD media, or at least give us a way to manually select the file and force UHD .bif generation.

 

[ExpedientFalcon 6]

I personally find option 3 to be the most pleasing, as option 4 looks oversaturated to me.

But I think my true preference would be for the thumbnails to match whatever tonemapping settings Emby would use to playback on a non-HDR device. i.e. I want the thumbnails and Emby's playback to look similar.

Edited April 14, 2022 by ExpedientFalcon

[cwills75 13]

2 minutes ago, ExpedientFalcon said:

I personally find option 3 to be the most pleasing, as option 4 looks oversaturated to me.

But I think my true preference would be for the thumbnails to match whatever tonemapping settings Emby would use to playback on a non-HDR device. i.e. I want the thumbnails and Emby's playback to look similar.

Yeah, everyone will have their own preference.  Option 4 matched my actual output very close in every image I looked at, but everyone's displays are set differently.