Jump to content

Tone-mapping in transcoding HDR for playback on SDR screens??


griffindodd
Go to solution Solved by Luke,

Recommended Posts

Here's one thought on this. We have the option of multiversioning, so we can have both HDR and SDR versions. But having the server intelligently choosing which to play in any given scenario, is not really in place. Example: if transcoding due to bandwidth is required, I believe what is presently happening is the server chooses the highest quality version to transcode from. If that is the HDR, then problem. But if when transcoding is needed with those versions available, I think the server should ignore the existence of an HDR version, and pick the closest SDR version to the requested output. Ignoring the HDR version would be best, because sometimes the 1080 version can have a higher bitrate than the HDR, and/or you have a 1080 HDR version. Of course, of there only exists and HDR version, then that would the default choice. The other caveat is that I have more and more TV shows in HDR, and have no intention of having both SDR and HDR versions. That would be a massive amount of data to store. For movies, I maintain both versions.

 

That's a very reasonable suggestion, and I would second that.

 

@@Luke

Link to comment
Share on other sites

 

I understand that story and that's the reason why we had put tone-mapping on our agenda - quite a while ago actually. But it was also clear that - when we'd do it - it needs to happen in hardware alongside all the other processing we're doing in hardware today. We've done a lot of preparation for this and it will be added soon.

 

But @@cryzis - how you set up your library is all your own decision of course, but considering movies and HDR: It's really not a necessity to have HDR versions for any video in your library. In fact there's just a small percentage where HDR really provides a significant visual benefit. 

 

I can only recommend to think twice regarding the selection of source videos - 10bit is not necessarily better than 8bit just because 10 is more than 8....

 

 

HDR is still in its infancy, I believe that production quality of HDR content will improve across the industry pretty quickly. Thats just a guess but consumers really like it especially when its done right, might as well support it as early as possible :)

Link to comment
Share on other sites

HDR is still in its infancy, I believe that production quality of HDR content will improve across the industry pretty quickly. Thats just a guess but consumers really like it especially when its done right, might as well support it as early as possible :)

 

That's not my point. It's not that HDR content is of bad quality. The problem is that an automated conversion will never be as good as like the studios are doing when they release their official 8 bit versions.

 

There are humans that know exactly how certain scenes are supposed to look like and these humans apply the right conversion parameters - scene-by-scene. They know how each scene is supposed to look like. An automated conversion can only apply some generic algorithms but the result won't equal the hand-adjusted original release.

Having an HDR display show SDR content is not really a big loss as long as it's not about a lot of very light, very dark, or very single-colored scenes...

Link to comment
Share on other sites

A few comments:

 

Of course "is being worked on" didn't mean permanent ongoing work. But there has been a long chain of prerequisites, like: before doing X, we need Y and before Y, we need Z, but we can't do Z without doing Q as well and that depends on ....

 

But where I don't really agree much is declaring tone-mapping as an inevitable feature. Wherever people get their content from these days: There's typically a choice between getting regular (SDR) and HDR content. Hence it's easy to follow a simple rule:

  • When you have HDR displays, get HDR content
  • When you have SDR displays, get SDR content
  • When you have both, well - ideally get both

The only valid use case I see for a server-side tone-mapping feature is the third case: You primarily have HDR displays and sometimes you want to watch the same content on SDR displays without duplicating content in your library.

 

But all of you asking for server-side tone-mapping need to be aware that this is not (and will never be) a high-end feature. It's a workaround only to make things appear less bad. It will never achieve the same quality of the original 8 bit content that you can get in the first place.

 

Emby's mission is "your media anywhere" and that's why we try to handle this situation as good as possible....

 

...but if somebody's library and Emby server setup strongly depends on having a tone-mapping feature in place for regular use, then I must clearly say that this is not a very good plan.

Regarding choosing when to get HDR media I think its extraordinarily rare that anyone would have only HDR displays. My primary local screen is HDR as every new TV is, and so I add media with that in mind, however if that was my only display I wouldn't be using Emby to begin with, Kodi is a better fit for a single display setup. The entire point of Emby to me is to make my media available remotely, and across a wide variety of devices and internet connections, most of which will not be suitable for HDR, and so transcoding is absolutely critical to Emby being of any use to me. My remote viewers do not have HDR displays for the most part, or the bandwidth to allow direct play of these files to begin with.

 

You also mention the quality of tonemapping, I don't think that's particularly important as transcoding is already a large loss of quality. My remote users are usually transcoding down to around 4-5mbps, the video just has to look ok, not perfect. When quality is of concern I will go out of my way to direct play anyway.

 

At the moment the only option for me is to have multiple libraries and multiple versions of content, and while possible that just doesn't seem like a very graceful solution to me.

Emby currently has all my media in one place, accessible to all users at the best quality their current connection allows. Multiple libraries breaks that, as well as being inefficient in storage and time required to add files and keep everything organized.

 

I understand tonemapping isn't simple to implement, and can never be perfect, and I am able to wait for it, but I feel a lot of people in this forum dismiss HDR tonemapping as unimportant.

To me its a critical feature, inevitably required as more and more media becomes HDR, and something I currently view as broken rather than a missing feature (I would prefer Emby refused to transcode HDR and popped up a warning message than the current washed out effort that confuses remote users).

 

Its an important enough feature for me that it would drag me to a competitor if they implemented it first and Emby still didn't have any announced timeline for it.

  • Like 1
Link to comment
Share on other sites

That's not my point. It's not that HDR content is of bad quality. The problem is that an automated conversion will never be as good as like the studios are doing when they release their official 8 bit versions.

 

There are humans that know exactly how certain scenes are supposed to look like and these humans apply the right conversion parameters - scene-by-scene. They know how each scene is supposed to look like. An automated conversion can only apply some generic algorithms but the result won't equal the hand-adjusted original release.

Having an HDR display show SDR content is not really a big loss as long as it's not about a lot of very light, very dark, or very single-colored scenes...

 

 

I was a bit confused by your response, that makes sense. Either way the overall quality loss isn't a huge deal it more about the availability of the content. 

Link to comment
Share on other sites

That's a very reasonable suggestion, and I would second that.

 

@@Luke

 

 

What would be the official path to lobby for this change? Emby to pick the SDR stream when transcoding is required.

Link to comment
Share on other sites

Final Summary 

(for all who might be confused by or short on time to read all the text)

 

Tone-Mapping is on the Roadmap

 

It will be added soon.

 

(just not yet in the upcoming 4.4 release)

  • Like 11
Link to comment
Share on other sites

  • 1 month later...
Stahlreck

I don't want to keep blowing this up, since it will be done at some point and I'm glad for that. I just find this discussion rather interesting, so sorry for bumping it up ^^

 

Emby's mission is "your media anywhere" and that's why we try to handle this situation as good as possible....

 

...but if somebody's library and Emby server setup strongly depends on having a tone-mapping feature in place for regular use, then I must clearly say that this is not a very good plan.

 

Right now Emby has multiple Problems with HDR though doesn't it? If you do anything other than DirectPlay, HDR on Emby is worthless anyway no matter the display, since transcoding breaks HDR. If Emby would just retain the HDR data, a lot of problems would vanish already (for my case at least). Because at least on my Windows 10 PC with Emby Theater, the app has no problem displaying HDR content on my SDR display with decent colors. Seems like the tone-mapping (or whatever it does) just gets handled by the client. But as soon as you transcode it's broken.

I'm not sure if that's just something Windows or Emby Theater on PC generally can do and if other clients could do it too...but generally I always assumed the client handles the colors. I mean, you can play UHD Blu-Rays on non HDR TVs AFAIK, and they all usually are HDR by default...well at least I don't really know UHD Blu-Rays that don't have HDR, even for old movies that get remastered.

 

But anyway, if the server will handle the tone-mapping in the end that would work too. Even if it's never as good as just a regular SDR version, still better than having a "white filter" on top of the video when it gets transcoded. A good enough trade-off for keeping just one library IMO. Thanks at all the hard working devs @Emby! I'm really glad this is being looked at! :)

Edited by Stahlreck
Link to comment
Share on other sites

  • 1 month later...
mawazi

I understand this is being worked on, and the main complication must be finding a way to make it work more universally and within acceptable hardware requirements. Is that correct? It can't be the actual server-side ffmpeg transcoding that is the issue, since that is done with a video filtergraph that is fairly simple, and works well in most cases. Something along the lines of: ffplay.exe -i hdrfile.ext -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p. I used ffplay to test, so that eliminates the re-encoding stage, which will require additional hardware resources.

 

So, making it as 'turn-key' as possible with minimal hardware requirements, those are the stumbling blocks, not the actual ability of ffmpeg to perform the tone-mapping. Do I have it right?

 

Thanks. 

Link to comment
Share on other sites

 

 

So, making it as 'turn-key' as possible with minimal hardware requirements, those are the stumbling blocks, not the actual ability of ffmpeg to perform the tone-mapping. Do I have it right?

Correct, and working out the problems that will come up with each platform or device that we support, each hardware transcoding vendor, etc.

Link to comment
Share on other sites

  • 7 months later...
  • 2 weeks later...
RDHoworth

Hi,

I have been using Emby now for a couple of years, but have just re-installed Plex as they now have the feature to apply an HDR Tone map when transcoding. I have not yet tested this but am now in the process of trying it.

I would rather stay with Emby, but unlike earlier comments, I feel that HDR is now quite mainstream, but itr is common that many households have TVs and displays, but of those, only a few are currently able to display HDR.

Please advise when Emby will likely have this feature, as waiting years for this is not good.

Regards, Richard

Link to comment
Share on other sites

bandit8623
5 hours ago, RDHoworth said:

Hi,

I have been using Emby now for a couple of years, but have just re-installed Plex as they now have the feature to apply an HDR Tone map when transcoding. I have not yet tested this but am now in the process of trying it.

I would rather stay with Emby, but unlike earlier comments, I feel that HDR is now quite mainstream, but itr is common that many households have TVs and displays, but of those, only a few are currently able to display HDR.

Please advise when Emby will likely have this feature, as waiting years for this is not good.

Regards, Richard

hdr tonemapping on plex is only working on hardware fully on linux.  i have a windows server and my 10 core Xeon cant process the 4k hdr tonemapping.  plex must not be able to handle that many threads,  because my cpu only hits 30%  but it cant keep up.  

  • Like 2
Link to comment
Share on other sites

6 hours ago, RDHoworth said:

Hi,

I have been using Emby now for a couple of years, but have just re-installed Plex as they now have the feature to apply an HDR Tone map when transcoding. I have not yet tested this but am now in the process of trying it.

I would rather stay with Emby, but unlike earlier comments, I feel that HDR is now quite mainstream, but itr is common that many households have TVs and displays, but of those, only a few are currently able to display HDR.

Please advise when Emby will likely have this feature, as waiting years for this is not good.

Regards, Richard

@RDHoworth - Could you please provide a reference to the Plex feature you're talking about, how it's working and what it's doing exactly? 

Thanks

Link to comment
Share on other sites

RDHoworth

This is very interesting, but I would be happy with the relatively basic HDR10. I understand that Plex will require a significant processor to do this, but I am running a Ryzen 9 12 core with lots of memory, so should stand a chance. This situation is far in advance of Emby, which has not moved on this issue for a very significant time. Given that I moved to Emby because of their development expertise and reputation for listening and responding to their user base, I am really disappointed on the seemingly lack of progress or information.

Reading this thread, many other users are similarily disappointed.

 

Link to comment
Share on other sites

vdatanet
15 hours ago, softworkz said:

@RDHoworth - Could you please provide a reference to the Plex feature you're talking about, how it's working and what it's doing exactly? 

Thanks

Jellyfin performs tone mapping using this:

http://ffmpeg.org/ffmpeg-all.html#tonemap_005fopencl

This is an example:

/usr/lib/jellyfin-ffmpeg/ffmpeg -ss 01:05:21.000 -hwaccel vaapi -noaccurate_seek -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device opencl=ocl@va -hwaccel_device va -hwaccel_output_format vaapi -filter_hw_device ocl -i file:"/media/4k/Saga del Infinito [4K UHDrip][2160p][HDR][AC3 5.1-DTS 5.1 Castellano-True HD 5.1-Ingles+Subs][ES-EN]/Guardianes De La Galaxia II 4Krip2160.www.pctnew.org.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:1 -map 0:2 -map -0:s -codec:v:0 h264_vaapi -b:v 5360000 -maxrate 5360000 -bufsize 10720000 -profile:v:0 high -level 41  -force_key_frames:0 "expr:gte(t,3921+n_forced*3)" -vf "scale_vaapi=w=1920:h=802:format=p010:out_range=limited,hwmap,tonemap_opencl=format=nv12:primaries=bt709:transfer=bt709:matrix=bt709:tonemap=reinhard:desat=0:threshold=0.8:peak=0,hwmap=derive_device=vaapi:reverse=1" -start_at_zero -vsync -1 -codec:a:0 aac -ac 6 -ab 640000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 1307 -hls_segment_filename "/var/lib/jellyfin/transcodes/a1e70b5a627fd40b9662b03d0efcb3de%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/var/lib/jellyfin/transcodes/a1e70b5a627fd40b9662b03d0efcb3de.m3u8"

 

Link to comment
Share on other sites

sooty234

Yeah, Reinhard is the simplest solution, but BT.2390 is the best, and is now industry standard.

719944516_Screenshot2020-12-27123713.jpg.884db786cb10ea4d81c1b90715d45126.jpg

 

Edited by sooty234
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...