Jump to content

Tone-mapping in transcoding HDR for playback on SDR screens??


griffindodd
Go to solution Solved by Luke,

Recommended Posts

37 minutes ago, sooty234 said:

Oh I wasn't implying that the process is simple. Looking at the algorithms that haasn has written, it's very complicated. But my general point is that adding tone mapping to an already in place downscaling of 4k to 1080 and reduction of color from 10bit to 8bit shouldn't add a lot more processing.

I'm afraid, but that assumption is very wrong (I wish it wasn't).

In fact it requires a multiple of the processing power than like for scaling.

  • Thanks 1
Link to comment
Share on other sites

sooty234
2 hours ago, softworkz said:

I'm afraid, but that assumption is very wrong (I wish it wasn't).

In fact it requires a multiple of the processing power than like for scaling.

Hmmmm.... I'm willing to bet that by the time you finish, it will work better than what these guys are thinking.

Link to comment
Share on other sites

sooty234
2 hours ago, cayars said:

Did you notice the resize command I added?  Did you look at the resolution after it converts?

Do you understand what's being shown?

I don't have the energy to fully abuse you at the moment. Wait until I recharge....

  • Haha 1
Link to comment
Share on other sites

1 minute ago, sooty234 said:

I don't have the energy to fully abuse you at the moment. Wait until I recharge....

sooty234 I think you missed what was being shown or what we're saying.  Try processing a 10 bit file and note the speed of ffmpeg working with that.  Try a similar bitrate 8 bit file and note the speed of that.  What speed difference do you see so far between 8 and 10 bit? Now try what I showed when you invoke tone mapping against either of the above and watch the speed drop way off.  You can certainly resize the input as I showed but it's far more CPU intensive doing tone mapping vs just working with 10 bit files.

If you don't like the ffmpeg command line I used, make or use your own tone mapping command line and compare that to the 10 bit processing.  What kind of speed difference do you see now?

Try resizing the input to 480 and I bet it's still slower then just normal 10 bit processing to 1080.

 

Link to comment
Share on other sites

sooty234

*sigh* You always think that if someone doesn't agree with you, it's because they don't understand. And no amount of any one of us trying to show you that you don't actually understand it yourself, makes no difference. You're under the impression that I'm an idiot. So be it. I really don't care. You're like Trump. We're all just tired of you. 

Anyway, I was trying to configure the transcoding thread count in the server to show you what I mean, but it appears to be completely broken and doesn't work at all. It still uses all CPU threads. (Emby has so many issues...*sigh*)

Yes Carlo, I know what you're trying and failing to show. Yes, I do actually understand. But your mind is a brick wall, again. You can't just simply throw whatever at a standard build of ffmpeg. Softworkz is so far beyond us, just as the mpv devs I talk to are. These guys work some kind of magic. Writing new algorithms to fit their needs. Just wait, I have every confidence that Softworkz will eventually validate my basic point. 

I'm not wasting any more energy on you. 

Link to comment
Share on other sites

Refer to what softworkz also said that backs up what I showed you concerning tone mapping vs 10 bit processing.

3 hours ago, softworkz said:

I'm afraid, but that assumption is very wrong (I wish it wasn't).

In fact it requires a multiple of the processing power than like for scaling.

 

Link to comment
Share on other sites

sooty234
3 minutes ago, cayars said:

Refer to what softworkz also said that backs up what I showed you concerning tone mapping vs 10 bit processing.

 

Yes, I read it, and I don't think it does. Just wait....

Link to comment
Share on other sites

RDHoworth

Hi,

I am finding this thread disappointing on two points:

  1. To hear an Administrator state that many people do not have hardware to do this is tantamount to saying that Emby will never be progressed because some users are running this on low power NAS units or 20 year old hardware. I know that this is not true, but the attitude that you should not develop something because it takes a lot oh hardware is ridiculous.
  2. Luke had stated that Tone mapping would appear in the beta within 1 - 2 weeks, around 2 weeks ago. However, this has not happened, and this conversation does not address the question at all. Discussing the difficulties is interesting, however, after several years work, I would have thought that we would have some idea on when, especially as your competitors (e.g. Plex) already claim to have this.

Hope to see a more useful response.

 

Richard

Link to comment
Share on other sites

sooty234

I am of the belief that this is a very low priority, to the point that they would prefer not to spend any time on HDR tone mapping. Extended delays and excuses. It's very difficult not to be annoyed.... so I'm just going to be annoyed. The attitude toward this endeavor from the emby people, is very poor. And Carlo Trump makes me want to shoot myself in the face. So much of emby is broken, and is just left to die. The emby people have lost touch with their user base, and it shows...

Link to comment
Share on other sites

vdatanet

This is transcoding and tonemapping using JF (i5 target 10 Mbits):

frame=  152 fps= 29 q=-0.0 size=    6144kB time=00:00:06.56 bitrate=7664.3kbits/s speed=1.26x    
frame=  170 fps= 30 q=-0.0 size=    6144kB time=00:00:07.31 bitrate=6883.4kbits/s speed=1.27x    
frame=  188 fps= 30 q=-0.0 size=    6144kB time=00:00:08.05 bitrate=6245.4kbits/s speed=1.28x    
frame=  206 fps= 30 q=-0.0 size=    6144kB time=00:00:08.80 bitrate=5715.0kbits/s speed=1.29x    
frame=  220 fps= 30 q=-0.0 size=   11264kB time=00:00:09.48 bitrate=9725.4kbits/s speed=1.29x    
frame=  236 fps= 30 q=-0.0 size=   11264kB time=00:00:10.06 bitrate=9168.8kbits/s speed=1.28x    
frame=  251 fps= 30 q=-0.0 size=   12544kB time=00:00:10.70 bitrate=9600.2kbits/s speed=1.27x    
frame=  265 fps= 29 q=-0.0 size=   12544kB time=00:00:11.32 bitrate=9075.4kbits/s speed=1.26x    
frame=  281 fps= 30 q=-0.0 size=   12544kB time=00:00:11.94 bitrate=8604.2kbits/s speed=1.25x    
frame=  292 fps= 29 q=-0.0 size=   12544kB time=00:00:12.51 bitrate=8208.4kbits/s speed=1.24x    
frame=  305 fps= 29 q=-0.0 size=   12544kB time=00:00:12.94 bitrate=7938.8kbits/s speed=1.22x    
frame=  308 fps= 27 q=-0.0 size=   12544kB time=00:00:13.07 bitrate=7860.5kbits/s speed=1.16x    
frame=  320 fps= 27 q=-0.0 size=   12544kB time=00:00:13.58 bitrate=7564.3kbits/s speed=1.15x    
frame=  337 fps= 27 q=-0.0 size=   12544kB time=00:00:14.28 bitrate=7192.1kbits/s speed=1.16x 

But if target is 50 Mbits:

frame=   85 fps=6.8 q=-0.0 size=   16128kB time=00:00:03.85 bitrate=34254.8kbits/s speed=0.307x    
frame=   88 fps=6.7 q=-0.0 size=   21760kB time=00:00:03.94 bitrate=45208.7kbits/s speed=0.301x    
frame=   92 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.07 bitrate=43787.3kbits/s speed=0.298x    
frame=   95 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.17 bitrate=42686.3kbits/s speed=0.294x    
frame=   98 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.34 bitrate=41007.1kbits/s speed=0.295x

If target is 10/15 Mbits I can stream 2/3 sessions. As I increase target bitrate, makes transcoding and tonemapping harder. For me it makes no sense transcode to that high bitrate, in that situation I can direct play, so I don't need transcoding, but using low bitrates is useful. Ideal for remote users in my family with limited bandwidth and avoid having multiple versions of the same media.

Edited by vdatanet
  • Like 1
Link to comment
Share on other sites

@sooty234 It's not a low priority and we're trying to get it into the next beta and get it out as quick as it makes sense to do.

@vdatanet yea you can see how performance drops off considerably as the bitrate goes up. If you don't mind me asking what OS are you running and what GPU?

Something to keep in mind, I previously said CPU tone mapping and with CPU only I can do the speeds you're seeing with the 50Mbit test with an i7.  Where I think things get interesting is with pascal or turing Nvidia GPUs using tonemap_opencl and the latest Nvidia drivers and opencl support of course.  That smokes what a CPU alone can do.  I'm assuming the speeds your showing above are using hw acceleration correct?

What you notice in the speed drop off are what we need to try and handle better than what you see with JF.  Obviously the 50 Mb would be buffering badly.

 

Link to comment
Share on other sites

sooty234
9 minutes ago, vdatanet said:

This is transcoding and tonemapping using JF (i5 target 10 Mbits):


frame=  152 fps= 29 q=-0.0 size=    6144kB time=00:00:06.56 bitrate=7664.3kbits/s speed=1.26x    
frame=  170 fps= 30 q=-0.0 size=    6144kB time=00:00:07.31 bitrate=6883.4kbits/s speed=1.27x    
frame=  188 fps= 30 q=-0.0 size=    6144kB time=00:00:08.05 bitrate=6245.4kbits/s speed=1.28x    
frame=  206 fps= 30 q=-0.0 size=    6144kB time=00:00:08.80 bitrate=5715.0kbits/s speed=1.29x    
frame=  220 fps= 30 q=-0.0 size=   11264kB time=00:00:09.48 bitrate=9725.4kbits/s speed=1.29x    
frame=  236 fps= 30 q=-0.0 size=   11264kB time=00:00:10.06 bitrate=9168.8kbits/s speed=1.28x    
frame=  251 fps= 30 q=-0.0 size=   12544kB time=00:00:10.70 bitrate=9600.2kbits/s speed=1.27x    
frame=  265 fps= 29 q=-0.0 size=   12544kB time=00:00:11.32 bitrate=9075.4kbits/s speed=1.26x    
frame=  281 fps= 30 q=-0.0 size=   12544kB time=00:00:11.94 bitrate=8604.2kbits/s speed=1.25x    
frame=  292 fps= 29 q=-0.0 size=   12544kB time=00:00:12.51 bitrate=8208.4kbits/s speed=1.24x    
frame=  305 fps= 29 q=-0.0 size=   12544kB time=00:00:12.94 bitrate=7938.8kbits/s speed=1.22x    
frame=  308 fps= 27 q=-0.0 size=   12544kB time=00:00:13.07 bitrate=7860.5kbits/s speed=1.16x    
frame=  320 fps= 27 q=-0.0 size=   12544kB time=00:00:13.58 bitrate=7564.3kbits/s speed=1.15x    
frame=  337 fps= 27 q=-0.0 size=   12544kB time=00:00:14.28 bitrate=7192.1kbits/s speed=1.16x 

But if target is 50 Mbits:


frame=   85 fps=6.8 q=-0.0 size=   16128kB time=00:00:03.85 bitrate=34254.8kbits/s speed=0.307x    
frame=   88 fps=6.7 q=-0.0 size=   21760kB time=00:00:03.94 bitrate=45208.7kbits/s speed=0.301x    
frame=   92 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.07 bitrate=43787.3kbits/s speed=0.298x    
frame=   95 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.17 bitrate=42686.3kbits/s speed=0.294x    
frame=   98 fps=6.7 q=-0.0 size=   21760kB time=00:00:04.34 bitrate=41007.1kbits/s speed=0.295x

If target is 10/15 Mbits I can stream 2/3 sessions. As I increase target bitrate, makes transcoding and tonemapping harder. For me it makes no sense transcode to that high bitrate, in that situation I can direct play, so I don't need transcoding, but using low bitrates is useful. 

There we go. Faster than real-time tone-mapped transcoding on an i5 CPU at 10Mb/s. Just as I predicted. Thanks @vdatanet

Link to comment
Share on other sites

Just now, sooty234 said:

There we go. Faster than real-time tone-mapped transcoding on an i5 CPU at 10Mb/s. Just as I predicted. Thanks @vdatanet

Highly doubt this is CPU only and will need hw/GPU support for decent bitrate handling.

Link to comment
Share on other sites

sooty234
1 minute ago, cayars said:

Highly doubt this is CPU only and will need hw/GPU support for decent bitrate handling.

GPU tone mapping isn't supported in JF or Plux, yet.

Link to comment
Share on other sites

RDHoworth

Again, why are we getting hung up on what hardware is needed? If we have the hardware, then we should be able to use it, if not, then that feature would not be available. Many people who use Emby are enthusiasts who want the best in Home Cinema, and not just cater to the lowest common denominator. Many users have used GPU acceleration for many years, and it is clear that this provides far more efficient transcoding capabilities than CPU only. 

  • Like 1
Link to comment
Share on other sites

sooty234
3 minutes ago, cayars said:

They BOTH make use of HW acceleration to assist the process which is a lot faster than CPU.

https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/
https://github.com/jellyfin/jellyfin/pull/3442

*sigh* There you go Trumping again..... You singlehandedly make me want to completely abandon emby. I'm just so damn tired of you. 

Link to comment
Share on other sites

vdatanet
15 minutes ago, cayars said:

Highly doubt this is CPU only and will need hw/GPU support for decent bitrate handling.

I'm using an i5 8600 running on Linux. If I disable hw transcoding in JF then tonemapping is disabled (I guess there's no software transcoding), if I increase target bitrate to 20 Mbps, then I get a performance drop, for me using JF the limit is at 15 Mbps (using Plex that limit is much higher and supports software transcoding)

frame=  112 fps= 19 q=-0.0 size=   10240kB time=00:00:04.94 bitrate=16963.8kbits/s speed=0.847x    
frame=  121 fps= 19 q=-0.0 size=   11520kB time=00:00:05.30 bitrate=17779.2kbits/s speed=0.838x    
frame=  131 fps= 19 q=-0.0 size=   11520kB time=00:00:05.69 bitrate=16582.6kbits/s speed=0.831x    
frame=  143 fps= 19 q=-0.0 size=   11520kB time=00:00:06.18 bitrate=15263.1kbits/s speed=0.838x    
frame=  154 fps= 20 q=-0.0 size=   11520kB time=00:00:06.69 bitrate=14095.9kbits/s speed=0.849x    
frame=  164 fps= 19 q=-0.0 size=   11520kB time=00:00:07.12 bitrate=13252.6kbits/s speed=0.844x    
frame=  175 fps= 20 q=-0.0 size=   16640kB time=00:00:07.56 bitrate=18009.6kbits/s speed=0.845x    
frame=  182 fps= 19 q=-0.0 size=   16640kB time=00:00:07.86 bitrate=17327.4kbits/s speed=0.824x    

 

Edited by vdatanet
Link to comment
Share on other sites

1 minute ago, RDHoworth said:

Again, why are we getting hung up on what hardware is needed? If we have the hardware, then we should be able to use it, if not, then that feature would not be available. Many people who use Emby are enthusiasts who want the best in Home Cinema, and not just cater to the lowest common denominator. Many users have used GPU acceleration for many years, and it is clear that this provides far more efficient transcoding capabilities than CPU only. 

Nobody is hung up on the hardware and we are not catering to the lowest common denominator.  Not sure where that came from.  But at the same time a low to mid end system without a GPU isn't likely going to be able to use this without an upgrade. If you can already transcode high bitrate 4K media then your system is a candidate for this. If your system AS IS right now can't transcode 4K media then with tone mapping there is virtually no chance it will work.

  • Like 1
Link to comment
Share on other sites

vdatanet
Just now, cayars said:

Nobody is hung up on the hardware and we are not catering to the lowest common denominator.  Not sure where that came from.  But at the same time a low to mid end system without a GPU isn't likely going to be able to use this without an upgrade. If you can already transcode high bitrate 4K media then your system is a candidate for this. If your system AS IS right now can't transcode 4K media then with tone mapping there is virtually no chance it will work.

Well that's true, we can't pretend to transcode using a Pi, if someone requires to transcode and tonemap he will obviously need the right hardware.

  • Like 1
Link to comment
Share on other sites

RDHoworth

Really, Who cares about software only encoding? It seems to me that you are just trying to justify none delivery in this case. You are getting hung up over hardware to justify your hard to defend position. As sooty234 has commented, please listen to your user base.

 

Link to comment
Share on other sites

sooty234

This is s**t. Emby is off the rails. Too much has been trivialized and forgotten. It's gone the way of all other software. The development is no longer in keeping with it's user base. There's nothing left to be said here...

Link to comment
Share on other sites

5 minutes ago, vdatanet said:

 If I disable hw transcoding in JF then tonemapping is disabled (I guess there's no software transcoding)

Yes, that's exactly what I'd expect to happen. But even with HW assist JF is struggling. Plex is a bit better but not much in this regard.

1 minute ago, RDHoworth said:

Really, Who cares about software only encoding? It seems to me that you are just trying to justify none delivery in this case. You are getting hung up over hardware to justify your hard to defend position. As sooty234 has commented, please listen to your user base.

Unfortunately a lot of people do software only encoding on Emby Servers. Maybe the NAS they have has no support for a GPU or they have no free slots. A lot of people also have older GPUs as well that work fine for H.264 encoding but not for H.265 (think GTX 750 class) and until now didn't really have a need to switch out what already works.  This is what we mean, that tone mapping likely won't work for them (almost certainly will not).  BUT that doesn't mean we're holding back on it.  If they want tone mapping they'll just have to upgrade the hardware properly to support it most likely.

The only reason we got talking about CPU only was because a certain person commented that tone mapping was very similar to working with 10 bit files and that's just not true and I gave an ffmpeg line he could use to prove it either way for himself.  Instead he took the conversation in a different route.

But almost certainly without monster CPU(s) you will need a GPU that's compatible with tone mapping to make use of the feature when released. Nothing is released yet so it's all premature conjecture right now based on what we know thus far.

Link to comment
Share on other sites

9 hours ago, RDHoworth said:

is tantamount to saying that Emby will never be progressed because some users are running this on low power NAS units or 20 year old hardware. I know that this is not true, but the attitude that you should not develop something because it takes a lot oh hardware is ridiculous.

Hi.  I'm sorry you read that but that is not at all what we said.  In fact, I stated just the opposite.

We simply want to set proper expectations so that people understand if the feature doesn't work on the hardware they have.  I'm pretty confident we are going to have the absolutely best performing HDR-SDR conversion out there - but that still may mean it needs adequate hardware to work in real time.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...