Washed colors in 4K transcoded to HD / SD

[ZiemaF 0]

Hi,

Is there any way to make videos in 4K HDR that are transcoded HD or SD did not have a washed out colors?
No matter if I play the movie through a browser (Chrome, Edge), the EMBY application for Windows, or on a POCO F2 Pro or LG G6 phone (HDR support), the transcoded material will always have such washed out colors.

Is there any way to do that? Because now practically the movies I have in 4K are not to be watched.

Thank you for your help
Ziema

 

[Carlo 4560]

At present you do not want to transcode 4K HDR material.

In the future this may be possible but for now it's much better to have both a 4K library and a 1080/ non 4K library.

[crusher11 1101]

It's not about resolution, it's about HDR. 

[ZiemaF 0]

This is what I expected after I searched the entire forum.
At the moment there are no (even test) solutions to this problem?

[PenkethBoy 2068]

No - tone mapping is something coming in the future - no eta yet - not easy to do well

so having 1080p content for those situations where transcoding is required for 4k - is the best solution for now - as 4k transcoding taxes even top end hardware

[crusher11 1101]

Even the major movie studios have trouble tone mapping at times. Many remastered Blu-Rays accompanying 4K UHD releases of catalog films have issues. 

[ZiemaF 0]

Transcoding itself is not so terrible, on the Intel i5 8 gen it gets about 90 frames/s from 4k to 720p.
Only this color is missing

[vdatanet 1617]

1 hour ago, ZiemaF said:

Transcoding itself is not so terrible, on the Intel i5 8 gen it gets about 90 frames/s from 4k to 720p.
Only this color is missing

Transcoding is not the issue, the issue is tone-mapping HDR to SDR while transcoding. It's on Emby's radar, but as far as I know, it's a complex matter.

[ZiemaF 0]

1 hour ago, vdatanet said:

Transcoding is not the issue, the issue is tone-mapping HDR to SDR while transcoding. It's on Emby's radar, but as far as I know, it's a complex matter.

I guess that's the way it is, Plex hasn't even dealt with it yet. I installed Plex especially for testing.
By the way, I remembered how much I dislike their Android applications.

[ZiemaF 0]

You can see that Plex wants to do something about it.

https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/?utm_source=Plex&utm_medium=email&utm_content=tone_mapping_button&utm_campaign=Newsletter_November_23_2020_NonPP

[Carlo 4560]

And if running on Windows for example it's mostly done in software and a 4K transcode won't work on anything but the fastest CPUs.

It's a start but still has quite a ways to go.  I'd still prefer to have 4K direct play and to choose a 1080p version otherwise.

[vdatanet 1617]

11 hours ago, cayars said:

It's a start but still has quite a ways to go.  I'd still prefer to have 4K direct play and to choose a 1080p version otherwise

The version selection logic is not perfect. If we have a 1080p version at 25 Mbps and another 4K HDR at 45 Mbps and the connection limit is 10 Mbps, instead of taking the 25 Mbps and transcoding, it takes the 45 Mbps, this stopped me from using the multiversion in the past. Does the same logic still apply?

Manual selection version is not an option for my relatives. So for me, tone-mapping and have only one version is the best solution. 

It is only a matter of time, in Plex a simple i5 8600 processor, transcodes 4K content doing tone-mapping using hardware acceleration (Linux) and barely consumes 5% of the processor.

[Carlo 4560]

Why not 4K videos in their own library this way you can grant permission to those who can direct play them?

[vdatanet 1617]

44 minutes ago, cayars said:

Why not 4K videos in their own library this way you can grant permission to those who can direct play them?

Yes, that's the current solution. I am lazy to maintain two libraries, the idea of having a single library seems attractive to me. I guess sooner or later this will change. Anyway, it's time to wait a bit.

[rbjtech 5284]

I don't actually have any issues with using a 4K library.  Since using the multi-version plugin (allowing film grouping from multiple storage areas) - you can apply library permissions for '4K Users' - who then get to see the 4K/1080p/720p selection.  Remote users are not permissioned for 4K (thus, they do not see the option) so they will either transcode from the 1080p version, or directly play the 720p if they have low bandwidth.

Transcoding anything means quality loss and unpredictability in my book  - so even without the HDR/SDR problem - I would never want to transcode a 4K movie Remux to a remote low bandwidth stream, or even a local 1080p stream when for the sake of 10-30 Gbytes of storage, I could have a professionally authored version direct play.   

[vdatanet 1617]

6 hours ago, rbjtech said:

I don't actually have any issues with using a 4K library.  Since using the multi-version plugin (allowing film grouping from multiple storage areas) - you can apply library permissions for '4K Users' - who then get to see the 4K/1080p/720p selection.  Remote users are not permissioned for 4K (thus, they do not see the option) so they will either transcode from the 1080p version, or directly play the 720p if they have low bandwidth.

Transcoding anything means quality loss and unpredictability in my book  - so even without the HDR/SDR problem - I would never want to transcode a 4K movie Remux to a remote low bandwidth stream, or even a local 1080p stream when for the sake of 10-30 Gbytes of storage, I could have a professionally authored version direct play.   

One user can play 4K/1080p/720p. My father is a remote user, in his living room he can play 4K, in the kitchen he can play 1080p and in his bedroom 720p. I have never been able to automatically choose the most suitable version, choosing the version manually is not an option. For me, the most practical is to have a single version that is transcoded according to the capabilities of the device and network conditions.

Edited November 25, 2020 by vdatanet

[rbjtech 5284]

16 hours ago, vdatanet said:

One user can play 4K/1080p/720p. My father is a remote user, in his living room he can play 4K, in the kitchen he can play 1080p and in his bedroom 720p. I have never been able to automatically choose the most suitable version, choosing the version manually is not an option. For me, the most practical is to have a single version that is transcoded according to the capabilities of the device and network conditions.

If Emby correctly chose the most suitable version - would your opinion change ?

I'm not 100% sure how Auto selection works - but I do know that for my remote users (bandwidth limited), emby chooses the 720p or 1080p low bandwidth (what I call my 'Streaming Copy') over an available higher bandwidth 1080p version.  Remote users don't have the option of 4K - there is little point imho - given the bandwidth they have.

On the LAN - of course you should not be bandwidth restricted,  but I'm not sure if emby THEN looks at the device capability to pick the most suitable copy ?   If it doesn't (and relies on bandwidth alone) then I guess this is actually the source of the problem ...

I will do some more digging ....

Edited November 26, 2020 by rbjtech

[vdatanet 1617]

Just now, rbjtech said:

If Emby correctly chose the most suitable version - would your opinion change ?

I'm not 100% sure how Auto selection works - but I do know that for my remote users (bandwidth limited), emby chooses the 720p or 1080p low bandwidth (what I call my 'Streaming Copy') over an available higher bandwidth 1080p version.  Remote users don't have the option of 4K - there is little point imho. 

On the LAN - of course you should not be bandwidth restricted,  but I'm not sure if emby THEN looks at the device capability to pick the most suitable copy ?   If it doesn't (and relies on bandwidth alone) then I guess this is actually the source of the problem ...

I will do some more digging ....

Yes, my opinion would change if auto selection worked at 100%. I only have 1080p and 4K versions, both bluray rips. Let's say I have a 1080p version at 25 Mbps and a 4K version at 50 Mbps. If bandwidth limit is set at 20 Mbps, instead of transcode 1080p version, it transcodes 4K version.

[rbjtech 5284]

4 minutes ago, vdatanet said:

Yes, my opinion would change if auto selection worked at 100%. I only have 1080p and 4K versions, both bluray rips. Let's say I have a 1080p version at 25 Mbps and a 4K version at 50 Mbps. If bandwidth limit is set at 20 Mbps, instead of transcode 1080p version, it transcodes 4K version.

ok - I see the issue.  I don't have that issue, as the 4K version is not permissioned. 

@Luke / @ebr - is there any reason why in the scenario above, emby is using the 4K version to transcode from ?  What is the logic it is following ?  Maybe it thinks (with some merit) that transcoding from the better quality version would produce the best output - but with the HDR issue + CPU overhead, would it not make more sense to transcode from the lowest quality version ? 

[vdatanet 1617]

1 minute ago, rbjtech said:

ok - I see the issue.  I don't have that issue, as the 4K version is not permissioned. 

@Luke / @ebr - is there any reason why in the scenario above, emby is using the 4K version to transcode from ?  What is the logic it is following ?  Maybe it thinks (with some merit) that transcoding from the better quality version would produce the best output - but with the HDR issue + CPU overhead, would it not make more sense to transcode from the lowest quality version ? 

I reported that some time ago, the answer was, when transcoding is required for best results, the highest bitrate version is taken. That's true, but the problem in this argument is that HDR is not taken into account, then colors are washed.

[rbjtech 5284]

Hmmm - I don't necessarily agree with that logic - and I think it needs to be smarter than that.    It should use the 'better' quality version as opposed to the 'best' quality version - saving on cpu, disk I/O etc.  

If I had a 90 Mbit/sec 4K remux and could only stream at 5 Mbit, why on earth would I use this as the source if I had a 10Mbit 1080p version available to transcode from ?...

 

[Carlo 4560]

It could use some additional smarts like looking at the color space and resolution to pick the "best" file to use for transcoding!

[GrimReaper 4739]

Or maybe use first higher bitrate one than the limiting bandwith setting? 

[sooty234 266]

When transcoding, if there is an SDR version available, the server should ignore the existence of the HDR version. 

But for me, tone mapping is essential. I have a lot of (and growing) HDR shows. Multi-versioning so many videos would be insane. It would cost a small fortune in hard drives. Resolution is irrelevant. Transcoding 4k is easy, it's just decoding. That doesn't take a lot, the encoding is the heavy lifting, but it's getting knocked down to 1080 or 720, that's not a lot of work. The server will never encode to 4k, so that's not a concern. A lot of my HDR shows are 1080. We need the tone mapping.

[Carlo 4560]

I'm "insane" then, LOL as I've got some movies 3 ways: 1080p, 4K, 3D
I'll personally continue to do this UNLESS hardware tone mapping can rival professionally produced SDR in color and quality.

But I can certainly understand why many/most people would prefer 1 single file!