Tone-mapping in transcoding HDR for playback on SDR screens??

[softworkz 3335]

7 minutes ago, cryzis said:

Nvidia tone mapping is in no way "slow". My P2000 could handle 5 simultaneous 4k HDR transcodes. It was VRAM limited (1GB per stream, OpenCL needed this for best performance).

I should have said that it is not as fast as it could be when using CUDA kernels instead of OpenCL because OpenCL requires downloading and re-uploading the data for OpenCL and then downloading and re-uploading again for encoding.

PS: And that's why it needs so much memory.

Edited January 31, 2021 by softworkz

[Carlo 4330]

15 minutes ago, cryzis said:

My P2000 could handle 5 simultaneous 4k HDR transcodes.

Nice high end card btw.  Just curious if you happen to remember the format of that media and what the bitrate of the original files were?
Do you happen to recall what resolution it transcoded to (720 or 1080) and what bitrate?
Just curious.

[cryzis 7]

10 minutes ago, cayars said:

Nice high end card btw.  Just curious if you happen to remember the format of that media and what the bitrate of the original files were?
Do you happen to recall what resolution it transcoded to (720 or 1080) and what bitrate?
Just curious.

The files ranged between 35-65 Mbps , and they were transcoding to 1080.

[cryzis 7]

33 minutes ago, softworkz said:

I should have said that it is not as fast as it could be when using CUDA kernels instead of OpenCL because OpenCL requires downloading and re-uploading the data for OpenCL and then downloading and re-uploading again for encoding.

PS: And that's why it needs so much memory.

That is fair, and that is a large waste of resources, it just happened to be the only way to get it done out of the gate. I think if you guys end up with a CUDA filter solution that would be absolutely great, very curious how much reduction in VRAM usage that would cause. I am convinced that pascal + should offer pretty great results with obvious improvements in the touring+ NVENC.

As for intel, it more of what I would consider the "holy grail", it appears built in HW tone mapping support for QS will only available in ICL+ (currently only available in laptops). It does appear that KBL+ has some kind of hardware tone mapping support that is interesting as well. I holding out hope that we may end up seeing at least 5 simul tone mapped transcodes with this solution but maybe that is a little too ambitious.

 

[softworkz 3335]

10 minutes ago, cryzis said:

I think if you guys end up with a CUDA filter solution that would be absolutely great, very curious how much reduction in VRAM usage that would cause.

You'll soon be able to find out

11 minutes ago, cryzis said:

As for intel, it more of what I would consider the "holy grail", it appears built in HW tone mapping support for QS will only available in ICL+ (currently only available in laptops). It does appear that KBL+ has some kind of hardware tone mapping support that is interesting as well.

As you said, the Linux driver as hw tone mapping included for Gen7+. For Windows, we just don't know yet, how they will do it. But the dev teams are not totally separate, it might happen that it will be just the same, once it is exposed via libmfx. Meanwhile, the "holy grail" is more or less the question, how to do it in OpenCL with memory sharing.
Let's see, who will be able to deliver that...

[Sammy 735]

Nice news!

How much is a P2000?

Anyone want a GTX1060 6Gb or will it suffice for maybe 2 simultaneous transcodes?

[Carlo 4330]

If you're running a turing GPU with the unlock applied I'd not be in a hurry to sell/trade up until you've tested first.

[Sammy 735]

What's a "turing GPU"?

I have unlocked my GTX 1060.

[Carlo 4330]

The latest die used on Nvidia GPUs.

Think of it as what generation the GPU belongs to.

[Sammy 735]

Got it. Googled it. I'm on Pascal and I'll see how that works first before dropping more $$$ on a GPU.

[Carlo 4330]

The 1060s came out as both pascal and turring.  I made sure mine was a turing model when I purchased it.

Either way I'd just wait and test.  I'm sure people will be posting number of transcodes that can be done and we can maybe all test with an HDR demo/trailer so we can compare numbers.

[Sammy 735]

I won't run beta server on my everyday server and my beta server runs on a low powered i3 NUC..

[Carlo 4330]

59 minutes ago, Sammy said:

I won't run beta server on my everyday server and my beta server runs on a low powered i3 NUC..

For testing tone mapping you can stand up a portable version of Emby on the same machine as another instance!

load up a half dozen HDR 4K movies and test.

[Sammy 735]

16 minutes ago, cayars said:

For testing tone mapping you can stand up a portable version of Emby on the same machine as another instance!

load up a half dozen HDR 4K movies and test.

To do this do I need to put the beta on a usb drive and load it from there?

[Carlo 4330]

No, just unzip it to a new directory (not your current Emby).  I for example use c:\EmbyPortable

switch to the system directory and manually start the server EXE file.

This will create all the folders under (in my case) C:\EmbyPortable.

[cptlores 23]

22 hours ago, RDHoworth said:

I repeat that this is still being arrogant. Why should you define what is reasonable or not? If a user wants to run this on a Threadripper  running Linux just so that can watch a tone mapped film on a phone. (I realize I am exaggerating). Then that is reasonable for them.

Exactly. It is also matter of logistics and scale. When you have a large library (>100TB), just the storage requirements (drives, raid controller etc) for multiple files will have a higher cost then adding more CPU/GPU hardware for transcoding.

Edited February 1, 2021 by cptlores

[Carlo 4330]

Oh btw, you will need to config it on first use like any other new install.

I'd turn off remote use and turn off automatic port forwarding just to make sure it doesn't interfere in any way with ports used for you main Emby install.

These are asked/configured during setup.

[Sammy 735]

1 minute ago, cayars said:

Oh btw, you will need to config it on first use like any other new install.

I'd turn off remote use and turn off automatic port forwarding just to make sure it doesn't interfere in any way with ports used for you main Emby install.

These are asked/configured during setup.

Shall I use a different port than 8096 for it? For example, I use 8097 on my NUC to avoid conflicts.

[Carlo 4330]

1 minute ago, cptlores said:

Exactly. It is also matter of logistics and scale. When you have a large library (>100TB), just the storage requirements (drives, raid controller etc) for multiple files will have a higher cost then adding more CPU/GPU hardware for transcoding.

It is a matter of logistics but NOT for the reason you mentioned.  

You can add almost an infinite amount of storage rather easily but you can not do the same for upgrade to CPU/GPU as there are physical limits and diminishing returns on faster HW.

Think about it another way.  If you're going to drop say $1000 (cheap) on CPU/GPU upgrades that's about 45 to 50 TB of drive space which I'm sure would hold all the 1080 versions of your 4K material.

[Carlo 4330]

2 minutes ago, Sammy said:

Shall I use a different port than 8096 for it? For example, I use 8097 on my NUC to avoid conflicts.

sure

[CBers 6771]

3 minutes ago, Sammy said:

Shall I use a different port than 8096 for it? For example, I use 8097 on my NUC to avoid conflicts.

You can use 8096 if on different computers. 

Different ports if on the same computer. 

 

[Sammy 735]

Well, I tried it and the beta installed. Not recommended to roll back, right?

At any rate, I'll turn off automatic updates to be safe with the current beta I believe being pretty stable.

Any progress on LiveTV NOT transcoding so that CC's will be exposed on AndroidTV yet?

Edited February 1, 2021 by Sammy

[RanmaCanada 344]

5 hours ago, cptlores said:

Exactly. It is also matter of logistics and scale. When you have a large library (>100TB), just the storage requirements (drives, raid controller etc) for multiple files will have a higher cost then adding more CPU/GPU hardware for transcoding.

Currently at 108 TB.  Carry multiple version of movies.  If you can hold a regular copy, then a Director's cut, having a 4k copy and a 1080p copy is not that big of a deal.  Again it is far cheaper to upgrade your space than to transcode the movie every single time someone wants to watch it on unsupported hardware, currently.  Once 11th gen Intel desktop chips are out I'll be buying an stx system with an i5 and will be having that as my new Emby/Plex server.  With hardware tonemapping, something the size of 2 books will make a feasible server that I can just stash away in the woodwork and remote into it when needed.

[cptlores 23]

1 hour ago, RanmaCanada said:

Again it is far cheaper to upgrade your space than to transcode the movie every single time someone wants to watch it on unsupported hardware, currently.

Even using a 1080p source, almost all the sessions I serve does a transcode in some form anyways. Mostly because of bandwidth issues at user side (flaky 'auto' detect scheme does not help either). But also because it turns out 'normal' people are perfectly happy with watching stuff on phones and tablets etc.

But that's not really a problem, since you can get older server equipment really cheap. Currently running a Dual Xeon with 16/32 cores @ 2.6/3.6Ghz, and that (CPU's & mobo) cost me less then $400. (edit) And looking at prices today, it would have been ever cheaper now I see..

 

 

 

Edited February 2, 2021 by cptlores

[Luke 37060]

The donuts are being iced as we speak: