Tone-mapping in transcoding HDR for playback on SDR screens??

[Carlo 4330]

1 hour ago, softworkz said:

I don't mean live streaming, I mean live-encoding.

What's the difference between the two in reference to how you're speaking about this.

[RDHoworth 6]

10 hours ago, sooty234 said:

After years of requesting, and lagging far behind the other platforms. This has been repeatedly pushed back. Remuxed HEVC hasn't even been fixed! Years have passed, and absolutely nothing has been done! 

My text...is 100% appropriate. 

Thanks for the voice of reason. I fully agree with your comments and they are fully reasonable. It is quite disrepective of the developers that try to force users into behavior that they do not want to follow, e.g. To keep multiple copies of files. Emby supports many types of usage, and it is not the purpose to try to "force" users into a particular usage pattern just because of a purist view.

[Carlo 4330]

@RDHoworthEmby hasn't been trying to force users to have multiple copies of the media, but it was the only was to do this in a reasonable manner.

Emby wasn't ignoring tone mapping at all.  The tools used to transcode (ffmpeg) just didn't support it so we had to wait until the added basic support, got it debugged and working with GPUs.  It's only recently gotten to this point in the last couple of months.  Meanwhile softworkz has been working on enhancements to the basic ffmpeg so it will work even better when used with Emby.

There is a difference of "not listening" and "having your hands tied" and not being able to do something.

[sooty234 266]

1 hour ago, cayars said:

@RDHoworthEmby hasn't been trying to force users to have multiple copies of the media, but it was the only was to do this in a reasonable manner.

Emby wasn't ignoring tone mapping at all.  The tools used to transcode (ffmpeg) just didn't support it so we had to wait until the added basic support, got it debugged and working with GPUs.  It's only recently gotten to this point in the last couple of months.  Meanwhile softworkz has been working on enhancements to the basic ffmpeg so it will work even better when used with Emby.

There is a difference of "not listening" and "having your hands tied" and not being able to do something.

Now you're pouring gasoline on the fire! You must love being abused! I'm trying to be polite, but I may just throw that out of the window... 

For now, I'll just say you're so full of s**t!

[RanmaCanada 335]

1 hour ago, cayars said:

@RDHoworthEmby hasn't been trying to force users to have multiple copies of the media, but it was the only was to do this in a reasonable manner.

Emby wasn't ignoring tone mapping at all.  The tools used to transcode (ffmpeg) just didn't support it so we had to wait until the added basic support, got it debugged and working with GPUs.  It's only recently gotten to this point in the last couple of months.  Meanwhile softworkz has been working on enhancements to the basic ffmpeg so it will work even better when used with Emby.

There is a difference of "not listening" and "having your hands tied" and not being able to do something.

Problem is, users see something "new and shiny" and they expect the devs to just kowtow to their demands and make it happen.  Much like the person who tried to demand action on DolbyVision, something that has only been possible for 6 months and most of it is hidden behind proprietary chips and software.  There is a clear disconnect between the users understanding the technology and the feasibility of implementing it.  I'll be blunt, the people demanding action on this have obviously not read the pages and PAGES of people complaining and bickering and finger pointing on the Plex forums or the Jellyfin forums about them not being able to get the implemented tonemapping to work properly on their POS hardware.  As far as the users are concerned, it should just work, damn the technical side of it.  "my 1st gen xeon Plex server should be able to tone map in software".  The users just don't understand, and their ignorance of how everything works behind the curtains is a serious problem.

Plex supports tone mapping 100% in Linux (hardware Intel/Nvidia and software via pure horsepower).  Everything else is software only.  Do you know the passmark required to tone map in software, because it takes a minimum of 17k to convert 4k HEVC to 1080p h.264.  That means you need a minimum of a Ryzen 5 3600x to perform 1 transcode, and that is all your server can do before tone mapping is calculated.

The other issues that everyone here is overlooking, there is no standard for tone mapping.  NONE.  

[Carlo 4330]

3 minutes ago, sooty234 said:

Now you're pouring gasoline on the fire! You must love being abused! I'm trying to be polite, but I may just throw that out of the window... 

For now, I'll just say you're so full of s**t!

You can certainly think what you want but what I just said is fact and also the devs have said the same thing.

[Carlo 4330]

Well said @RanmaCanada.  Yes some people think this is super easy or not hard on the CPU and some have compared it to working with 10 bit which of course is not even close to reality. Tone mapping done correctly for a media server (not one off conversions) is very tricky to do RIGHT at better than real-time.

Emby is working on a solution that will work for different OSes and different GPUs and not just Linux. It's also looking at ways to extend transcoding in general in the future.

[sooty234 266]

*hands over a roll of TP* wipe your mouth...

[RanmaCanada 335]

1 hour ago, cayars said:

Well said @RanmaCanada.  Yes some people think this is super easy or not hard on the CPU and some have compared it to working with 10 bit which of course is not even close to reality. Tone mapping done correctly for a media server (not one off conversions) is very tricky to do RIGHT at better than real-time.

Emby is working on a solution that will work for different OSes and different GPUs and not just Linux. It's also looking at ways to extend transcoding in general in the future.

And that is why I suggested that the team limit it to Intel 11th Gen only, at least until something more reliable comes forward.  There are open-cl plugins that sometimes work, and someone is attempting to develop a Cuda based filter, but nothing is working 100%.  Intel will have it hard coded into their ASICS for quicksync.  https://edc.intel.com/content/www/us/en/products/performance/benchmarks/11th-generation-intel-core-tgl-u-and-evo-launch-2020/

And for those saying that softworkz has been sitting on their duff doing nothing, they've been actively trying to get Intel to get this implemented.

Stuff like this is out of the hands of the Dev team if the software that they use has not been built in house.  

And sooty, you're being extremely disrespectful and beyond rude.  Educate yourself on what is happening in the back end before you comment on things that are obviously far beyond your limited understanding.

[sooty234 266]

29 minutes ago, RanmaCanada said:

And sooty, you're being extremely disrespectful and beyond rude.  Educate yourself on what is happening in the back end before you comment on things that are obviously far beyond your limited understanding.

Ha! My disrespect is apt, as I've lost most of it and losing more. 

As for HDR and tone-mapping. I am quite well versed.

[softworkz 3326]

34 minutes ago, RanmaCanada said:

And that is why I suggested that the team limit it to Intel 11th Gen only, at least until something more reliable comes forward. 

Stay tuned, it won't be that bad.

[RanmaCanada 335]

3 hours ago, sooty234 said:

Ha! My disrespect is apt, as I've lost most of it and losing more. 

As for HDR and tone-mapping. I am quite well versed.

If you're so APT at it, then how about you get off your duff and create a plugin that will actually work and can be adopted industry wide.  Or is berating the dev team the only thing you can do?  Intel themselves have been working on it for years, so please, use your expertise to help them, as it's obvious only YOU can save them from their folly.  If a multi-billion dollar company is having issues solving it, it's obviously not something that can be done by a "rag tag" group of developers.

[sooty234 266]

1 hour ago, RanmaCanada said:

If you're so APT at it, then how about you get off your duff and create a plugin that will actually work and can be adopted industry wide.  Or is berating the dev team the only thing you can do?  Intel themselves have been working on it for years, so please, use your expertise to help them, as it's obvious only YOU can save them from their folly.  If a multi-billion dollar company is having issues solving it, it's obviously not something that can be done by a "rag tag" group of developers.

Oh dear, how little you know. Go and sit quietly in a corner. You're all stuck on minutia. That link to Softworkz' request is meaningless. A f**king year! It's only Intel. Only one variant of hardware acceleration. Worthless! Successful high quality tone-mapping has been around for several years. Madshi and the mpv boys have this all dialed in. F**k Intel. You're all stuck with your thumbs up your a**. Adequate software tone-mapping should have been put in beta more than a year ago. And we could have spent a lot of productive time working out the bugs. Instead, it got put aside because why, you're waiting on the Intel douche-nozzles? Muppets! Now what we're going to have is a bottleneck of testing, bouncing between hardware and software. Yes, many CPUs at first won't have the strength to keep up in real-time, but that doesn't matter. This is what beta is for. Honing the software and getting the options and mechanics dialed in. This is a f**king debacle, with a comedy of errors. 

It's a disgrace!

 

I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... 

Edited January 28, 2021 by softworkz

[sooty234 266]

21 minutes ago, sooty234 said:

Oh dear, how little you know. Go and sit quietly in a corner. You're all stuck on minutia. That link to Softworkz' request is meaningless. A f**king year! It's only Intel. Only one variant of hardware acceleration. Worthless! Successful high quality tone-mapping has been around for several years. Madshi and the mpv boys have this all dialed in. F**k Intel. You're all stuck with your thumbs up your a**. Adequate software tone-mapping should have been put in beta more than a year ago. And we could have spent a lot of productive time working out the bugs. Instead, it got put aside because why, you're waiting on the Intel douche-nozzles? Muppets! Now what we're going to have is a bottleneck of testing, bouncing between hardware and software. Yes, many CPUs at first won't have the strength to keep up in real-time, but that doesn't matter. This is what beta is for. Honing the software and getting the options and mechanics dialed in. This is a f**king debacle, with a comedy of errors. 

It's a disgrace!

 

I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... I will learn how to write nice and friendly posts... 

No I won't! I will perfect being a ferocious A-hole  

[arrbee99 1551]

1 minute ago, sooty234 said:

No I won't! I will perfect being a ferocious A-hole  

You don't want to know what kind of mental picture that brings to mind...

[sooty234 266]

Just now, arrbee99 said:

You don't want to know what kind of mental picture that brings to mind...

Haha... well I did choose my words...

[Carlo 4330]

 

1 hour ago, sooty234 said:

Madshi and the mpv boys have this all dialed in. F**k Intel. You're all stuck with your thumbs up your a**. Adequate software tone-mapping should have been put in beta more than a year ago.

You can keep saying the same base less information to the cows come home but anyone who follows this knows better and knows facts. It's not a simple process at all to do this while transcoding and surely not anything like just tone mapping on the display side like what takes place in mpv.  Rather simple algorithms can do that.

The above is pretty trivial task compared to making this a filter for ffmpeg. Not understanding this process is the source of problem why you think it's trivial and should have been in the software a year ago.

The problem even now while doing tone mapping while transcoding is the combinations of things taking place.  For example you can DECODE on the GPU, resize on CPU, run filters in GPU, tonemap in CPU, run additional filters in GPU and then encode in GPU.  You're passing a lot of data back and forth which causes bus IO and requires CPU use. This can still nail the CPU even when the GPU is only 20% used.  On the other hand if you just want to DISPLAY the data it's easy to tone map this and doesn't take a lot of resources.

When chip manufactures can do this on chip in the GPU you remove a lot of system resources needed to do this and can just hand off the process to the GPU while controlling the process but allow everything to take place there on the GPU.

Some filters in ffmpeg can't use all cores available and some will only run in 1 or 2 cores and this too creates a severe bottleneck. This is exactly the problem a year ago and why it wasn't really possible to do for REAL-TIME transcoding.  Only recent drivers from video companies support enough HW acceleration functions to make this worthwhile adding to ffmpeg in a meaningful way.

[sooty234 266]

9 minutes ago, cayars said:

 

You can keep saying the same base less information to the cows come home but anyone who follows this knows better and knows facts. It's not a simple process at all to do this while transcoding and surely not anything like just tone mapping on the display side like what takes place in mpv.  Rather simple algorithms can do that.

The above is pretty trivial task compared to making this a filter for ffmpeg. Not understanding this process is the source of problem why you think it's trivial and should have been in the software a year ago.

The problem even now while doing tone mapping while transcoding is the combinations of things taking place.  For example you can DECODE on the GPU, resize on CPU, run filters in GPU, tonemap in CPU, run additional filters in GPU and then encode in GPU.  You're passing a lot of data back and forth which causes bus IO and requires CPU use. This can still nail the CPU even when the GPU is only 20% used.  On the other hand if you just want to DISPLAY the data it's easy to tone map this and doesn't take a lot of resources.

When chip manufactures can do this on chip in the GPU you remove a lot of system resources needed to do this and can just hand off the process to the GPU while controlling the process but allow everything to take place there on the GPU.

Some filters in ffmpeg can't use all cores available and some will only run in 1 or 2 cores and this too creates a severe bottleneck. This is exactly the problem a year ago and why it wasn't really possible to do for REAL-TIME transcoding.  Only recent drivers from video companies support enough HW acceleration functions to make this worthwhile adding to ffmpeg in a meaningful way.

Shut up, donkey. I if want your opinion, I'll give it to you. And you still won't understand.

[Carlo 4330]

3 minutes ago, sooty234 said:

Shut up, donkey. I if want your opinion, I'll give it to you. And you still won't understand.

I'm sorry but you're the one putting down the software and company based on very faulty information and lack of knowledge on this particular subject.

[crusher11 850]

So...is it at all possible to get suspended/banned from this forum? I can't think of anywhere that allows constant abuse of the staff.

Relatedly, is there an 'ignore this poster' function?

 

[softworkz 3326]

29 minutes ago, crusher11 said:

s it at all possible to get suspended/banned from this forum?

For a long time, up until decades ago, it's been quite normal that humans with certain disabilities were publicly exposed and presented - on fairs or bustles allowing to watch these in exchange for just little money. Something like "woman without legs", small-grown men, a guy that doesn't feel pain.. etc. - a bit like a zoo with humans.
This had eventually been superseded and replaced by afternoon TV talk shows with the focus shifting a bit from physical to mental shortcomings. The mechanism remained unchanged, though: humans are often fascinated by  abnormal or cripple individuals, no matter whether physical or emotional.

Eventually, after those talk shows, came the internet - again providing plenty of ways for spectating even more human weirdness. Again - same mechanisms in place.

And so - here we are now, with our zoo having a new attraction 

[Luke 37008]

Quote

with our zoo

Speaking of the zoo, let's lighten the mood with one of my favorite BBT episodes:

 

[Carlo 4330]

On that note I think this is my favorite scene from TBBT. Leonard Nimoy's DNA

Bazinga

Edited January 28, 2021 by cayars

[Abobader 2941]

11 hours ago, crusher11 said:

So...is it at all possible to get suspended/banned from this forum? I can't think of anywhere that allows constant abuse of the staff.

Relatedly, is there an 'ignore this poster' function?

 

They are indeed, but we tried to give some time, if thing do not change, ban will apply.

We normally do not like to go by that route here unless it the last option.

[RanmaCanada 335]

2 hours ago, Abobader said:

They are indeed, but we tried to give some time, if thing do not change, ban will apply.

We normally do not like to go by that route here unless it the last option.

How many more chances do they get?  Multiple times in this thread they have been abusive to anyone who dares go against their uninformed opinion.  I myself have been angry at things that have happened, but I've never gone as far as this person has with their disrespect and personal attacks.  They also continue to sow misinformation and discord.