Problem transcoding when source file is h.265

[Carlo 4560]

Hardware is often less flexible with somethings compared to software encoding.

But my question regarding this for the 4K HDR media, did you like the transcoded video or was it washed out looking?

[muzicman0 84]

I will go back and look at it again, but I don't remember it being washed out.  But it was a while ago that I checked it, so who knows.  

[muzicman0 84]

yes, it is washed out.  I understand why.

[Luke 42077]

Ok, @softworkz will go over this and comment on it. Thanks.

[softworkz 5066]

Is it still about why hw decoders may behave differently from sw decoders?

The only other (serious!) advice I have is this: don't perform homemade transcodings, rather get another HD.

Regards,
softworkz

[muzicman0 84]

8 minutes ago, softworkz said:

Is it still about why hw decoders may behave differently from sw decoders?

The only other (serious!) advice I have is this: don't perform homemade transcodings, rather get another HD.

Regards,
softworkz

I currently have 30 TB of HDD space.  half of that is for backups.  Can you imagine how much space I would need if I didn't transcode these files????  Granted, some of the files are original disc rips (I don't transcode those), however, I have almost 7000 episodes of TV, and a ton of recorded movies, plus all the disc rips, etc.  Getting more HDD space is an option, but probably not a good one.

I guess it just is what it is.  They play back fine if I use software decoding (my homemade ones), and only have issues if transcoding (which translates to remote viewing) while using hardware accelerated decoding.  It could be something on the ROKU side of things, however, files that I transcoded to h.264 in the past play fine.

Maybe I will upgrade this PC to a more modern CPU to keep the CPU usage down.  I have a tendency to over build my PC's so they last a long time, and I am currently using a i7-4790k (built back when that CPU was top of the line).  I have been looking at upgrading to either a i7-10700k, or an i9-9900k.  It's no small invenstment on my part though, so I wanted to make sure nothing could be done in software, since other than this issue, the PC still works great.

[softworkz 5066]

Please understand that I cannot start investigating your whole chain: original file >> your transcode command >> result file >> emby ffmpeg operation

That is complex and requires significant work and is finally not really related to Emby.

One alternative option might be to use the DXVA2 hevc decoder (in case there is one). 

Second option: In the future we might implement a slightly different Nvidia decoding acceleration where ffmpeg performs the bitstream parsing and the hardware just decodes the raw video stream. So far, we haven't been able to decide whether to make that switchable or not (risk). It might take a while, but then there are chances(!) that it could work.

[muzicman0 84]

I get it.  Thanks.

[Carlo 4560]

53 minutes ago, muzicman0 said:

I currently have 30 TB of HDD space.  half of that is for backups.  Can you imagine how much space I would need if I didn't transcode these files????  Granted, some of the files are original disc rips (I don't transcode those), however, I have almost 7000 episodes of TV, and a ton of recorded movies, plus all the disc rips, etc.  Getting more HDD space is an option, but probably not a good one.

Not much more at all, as you're still a "young pup" storage wise.   If we assumed your 15 TB of used storage was all 4K and you added 1080 versions of all movies you would need less than 1/5 the space for the HD version or roughly 3 GB to have matching 4K/HD versions which makes the system much easier to use and raises the quality of viewing as more videos can be direct played as "produced" for optimum quality.

If you want to get into 4K video and want to have a nice size library you will need a LOT MORE storage.  4K sucks up the space quickly and honestly isn't needed for every movie or show. Storage is getting cheaper and cheaper all the time but is a drop in the bucket compared to the cost of the media.

But you don't have to hoard things and can get by on limited storage.  Just do what Amazon Prime, Hulu and Netflix do and rotate content. If you want to hoard a lot of 4K movies than prepare to purchase a 10 TB drive every paycheck. Ask me how I know, LOL

 

[muzicman0 84]

It's not the  movies that I have a problem with (typically)...I think I have maybe 3 UHD discs, so having 1080p versions of those is no issue.  but TV episodes on the other hand...that would take up a decent amount of space.

I transcode to h.265 so that I can keep the native resolution of the episode.  Even with that, I went from about 600 MB per episode (480p h.264) to 2 GB with h.265 per episode.  But that is a savings of anywhere between 50 and 75% (original file is 4-8 GB).

Now if I could just figure out how to make Nvidia encodes of h.265 not have banding on the blacks, I would be super happy!

Edited September 2, 2020 by muzicman0

[Carlo 4560]

I do my encoding using only software as I get smaller files, more control and better quality.  Takes longer however.

[muzicman0 84]

yes, I used to use software encoding, but during heavy 'new episode' TV season, I can encode 4-5 episodes per night (automated), and with h.265, it can take hours per episode with my current setup.  Plus the heat it generates with the PC was a concern for CPU life span.

[PenkethBoy 2068]

i'm sure some will disagree but i find with the settings that work for me that re-encoding H264 etc content to H265 works fine and if you experiment it can give you just as good a result as software re-encoding

You have the same level of control (options) and although the result may not be of the same size - the time it takes (considerably less ) offsets the saving in size and the power demand i believe is less as well.

An average tv epidsode say 45 mins takes 7-8mins and the average movie approx 15-20 mins rather than hour(s) with CPU only - via Nvidia GPU - in my case an older 980TI - as i am sure the newer 10 and 20 series cards would be faster - and possibly better quality output - but cant test that myself.

I have done well over 15,000 videos of various types and 99.9% of those have produced a more than acceptable output. The more testing you do upfront the better to fully understand what can be achieved with ffmpeg and a moderately modern Nvidia card. (took about three months on and off to convert all of those videos)

With regard to 4k videos i leave those alone as the 908TI cant handle those unless they are 8bit that i want to downsize to 1080p

[Carlo 4560]

The reason I convert from H.264 to H.265 is space savings and network/bandwidth savings.  If I get subpar results from HW compared to SW I've defeated the purpose (for me).

[Happy2Play 9780]

Only saves bandwidth if h.265 direct plays/streams, if not transcoding doubles the bandwidth.

[PenkethBoy 2068]

7 minutes ago, cayars said:

The reason I convert from H.264 to H.265 is space savings and network/bandwidth savings.  If I get subpar results from HW compared to SW I've defeated the purpose (for me).

Space saving you get by default going from h264 to h265 - the saving can be big or small all comes down to settings - but a file 30% the size of the original is usually the result

SW may get you a smaller output but its not usually significantly different and the quality can also be worse - setting dependant again

thing is you dont get subpar results - unless you go with default settings - which is the same result you would get with SW and default settings

 

[Carlo 4560]

2 minutes ago, Happy2Play said:

Only saves bandwidth if h.265 direct plays/streams, if not transcoding doubles the bandwidth.

I wouldn't say double but it puts it back about where it started in the original H.264 file (only degraded by 2 conversions).

Yes of course you're correct but for me ever device I use can direct play H.265.  I only have 10 to 12 Mb upload from my server so h.265 is a blessing for me.

Everyone has different needs of course.  For me I'll trade slower one-time conversions to get high quality files using the least bitrate needed (vs HW).  Help with both streaming and of course storage.  I've got probably half a petabyte of h.265 encoded by software at this point so if I did that with HW I'll likely need 700 TB vs 500 TB for the same files.  Adds up.

[PenkethBoy 2068]

LOL 200gb extra i doubt thats even close to being correct

[edit]

yep not had enough coffee

TB it is - still a very high figure for the difference

Edited September 3, 2020 by PenkethBoy

[softworkz 5066]

Just a side note: @PenkethBoy is right: It's all about the settings.

The only reason why one could easily think that sw encoding quality would be much better than hw encoding, is a mismatch in Emby's default settings for either case.

(ultimately, sw libx265 is probably in fact superior to most hw encoders, but that's just the final few percent. The effect/difference that you're typically seeing is rather caused by the above)

[Carlo 4560]

1 minute ago, PenkethBoy said:

Space saving you get by default going from h264 to h265 - the saving can be big or small all comes down to settings - but a file 30% the size of the original is usually the result

SW may get you a smaller output but its not usually significantly different and the quality can also be worse - setting dependant again

thing is you dont get subpar results - unless you go with default settings - which is the same result you would get with SW and default settings

I get a pretty big difference in bitrate needed between SF and HW for the same quality output. There is a lot you can tune via software that you just can't do in a HW conversion.  

7 minutes ago, PenkethBoy said:

LOL 200gb extra i doubt thats even close to being correct

200 TB not GB.  That's a lot of storage!

6 minutes ago, softworkz said:

Just a side note: @PenkethBoy is right: It's all about the settings.

The only reason why one could easily think that sw encoding quality would be much better than hw encoding, is a mismatch in Emby's default settings for either case.

(ultimately, sw libx265 is probably in fact superior to most hw encoders, but that's just the final few percent. The effect/difference that you're typically seeing is rather caused by the above)

I'm not talking doing this inside of Emby.  I'm talking about outside of Emby getting files ready before adding them to Emby.  So the sky is the limit on settings as they can be completely customized.  Of course it's about settings but some things can't be done in HW that can be done in SW.  Some of this depends on the HW you have as well and mine is not the latest greatest so that will play into as well but with SW I always have the latest/greatest just maybe not the speed which is secondary to me over size and bitrate.

[Carlo 4560]

PenkethBoy, when you convert to h.265 are you doing 8 or 10 bit files?

I'm only doing 10 bit.

[PenkethBoy 2068]

8 bit as 980 does not do 10bit

[PenkethBoy 2068]

sw generally gives lower bit rates than HW for similar settings (the settings are not the same) - hence the lower files size - but IIRC as its been a while the lower bit rate sw re-encodes were inferior to the hw results - and no noticable diff at comparable bit rates

i wanted to do the process quickly so concentrated on the hw option - you can do all you need with the HW options - the fact sw has more is probably due to that was the only way to do it until the last few years - but i dont see any limitations in what options you have with HW

[Carlo 4560]

PenkethBoy, I do 10 bit encodes which gives a bit better compression.  As mentioned all my devices can handle this.

@softworkz, PenkethBoy or anyone else.  I'm willing to give HW another shot.  Been a while and my HW would need upgrading to "match" software.  So with that in mind if you were going to build a new machine or upgrade SOLELY for H.265 conversion  (10 bit) outside of Emby what would you get?  GPU (which one - consumer level) or a late model i7 with QuickSync?

Size and bitrate are much more important to me than speed as long as I have same quality. 

[muzicman0 84]

@PenkethBoy, have you figured out a decent way on your Nvidia encodes to get dark scenes to not have banding?  I have really good quality, other than mostly dark/black scenes.