In-Progress Transcode in H265

[syadnom 25]

23 minutes ago, softworkz said:

LOL. Their ffmpeg customization code is copied from Emby's and Plex' ffmpeg

uh, blatantly false.

[softworkz 5066]

21 minutes ago, syadnom said:

uh, blatantly false.

Well - you're talking to the wrong person. Or rather - the right one.
But I don't want to be unfair, so to be more precise: at least 80% of their ffmpeg patches are copied, and a large percentage of their server code is still from Emby 3.5
What I can also say is that in the whole area of transcoding code, Emby 4.7 is by a magnitude more different from Emby 3.5 than the current code of that fork has changed). 

[softworkz 5066]

Another fun fact: Emby's ffmpeg has by far the most customizations from all.

[japtain_cack 6]

4 hours ago, softworkz said:

LOL. Their ffmpeg customization code is copied from Emby's and Plex' ffmpeg

Well, I can transcode in HEVC with jellyfin, can't with emby/plex, so obviously they've made some improvements. I realize that jellyfin is a fork, so it's all based on embys code really.

[softworkz 5066]

7 minutes ago, japtain_cack said:

Well, I can transcode in HEVC with jellyfin, can't with emby/plex,

I had tried to explain this a few posts above. Getting HEVC transcoding "somewhat working" is not a big deal and achieving coverage of that functionality for something like 50-80% of  cases could be achieved with moderate effort.

They can do this. It's free and developers are volunteers - nobody can blame them for anything not fully working.

But when WE do that, users have much higher expectations and we for ourselves, have a different set of standards under which new features are being released. Our users can expect that we are releasing features which are fully working. In that post above, I had tried to explain that the really hard part in this case are not those 0-80% but the last 20% until it would be working in the same perfect and reliable way like it is working now with H264.

After all: Why are users coming here and telling what others can do instead of happily using theirs?
I think in most cases, it's because they have these higher expectations that others just can't fulfill.

[japtain_cack 6]

35 minutes ago, softworkz said:

I had tried to explain this a few posts above. Getting HEVC transcoding "somewhat working" is not a big deal and achieving coverage of that functionality for something like 50-80% of  cases could be achieved with moderate effort.

They can do this. It's free and developers are volunteers - nobody can blame them for anything not fully working.

But when WE do that, users have much higher expectations and we for ourselves, have a different set of standards under which new features are being released. Our users can expect that we are releasing features which are fully working. In that post above, I had tried to explain that the really hard part in this case are not those 0-80% but the last 20% until it would be working in the same perfect and reliable way like it is working now with H264.

After all: Why are users coming here and telling what others can do instead of happily using theirs?
I think in most cases, it's because they have these higher expectations that others just can't fulfill.

I understand your reasoning, and don't disagree at all. I do believe Emby is a much more polished product than Jellyfin for obvious reasons. However, after using both, and having a better streaming experience with Jellyfin, using HEVC transcoding over the WAN, I'm willing to sacrifice a little polish/reliability with functionality.

My comparison of the two involved using telegraf/grafana to view time series data on my WAN connection and HAProxy connections. I also compared various platforms and apps, set them on 1Mbps and watched a 4k video. I had a clear increase in quality when using HEVC transcoding. I actually recorded bandwidth of around 1-2Mbps, when setting it to 1Mbps, but I'm sure that's normal.

When setting bandwidth limits to unlimited and forcing transcoding, I also noticed a bandwidth decrease. So even if the benefits are theoretically slight, they seem to have real world benefits.

Either that or the benefits of HEVC transcoding are truely miniscule and this must have something to do with jellyfin's implementation. I couldn't actually say for sure, but I do know, my parents/family/friends have reported a much better experience on Jellyfin. Less buffering and better quality (I globally limit remote stream to 3Mbps), but they did like Emby's UX better (I agree).

In the end, actually being able to reliability play a video, without constant pausing/buffering, is what makes my decision for me. I do prefer Emby though, so I'll be following it's development. 

Edited October 29, 2022 by japtain_cack

[softworkz 5066]

11 minutes ago, japtain_cack said:

So even if the benefits are theoretically slight, they seem to have real world benefits.

HEVC has benefits - there's no doubt about that. It's all about outweighing the benefits vs. the required effort.

17 minutes ago, japtain_cack said:

set them on 1Mbps and watched a 4k video

Sure. That's a case where the abilities of HEVC become very visible. I'm not sure though, whether 4k makes sense at all when you have just 1Mbps available.
It's a combination that Emby doesn't even use or allow to configure.

21 minutes ago, japtain_cack said:

Either that or the benefits of HEVC transcoding are truely miniscule and this must have something to do with jellyfin's implementation. I couldn't actually say for sure, but I do know, my parents/family/friends have reported a much better experience on Jellyfin. Less buffering and better quality

We take user issues serious, so please create a new topic, so we can take a look at it.
Let's not discuss this here, but two notes up-front: when there's buffering, it's usually either the server not transcoding fast enough or the target bandwidth isn't met or not calculated correctly. And with regards to quality: the default H.264 transcoding settings are set for moderate quality, it's not at the edge of what's achievable with H264.

[japtain_cack 6]

15 minutes ago, softworkz said:

HEVC has benefits - there's no doubt about that. It's all about outweighing the benefits vs. the required effort.

Sure. That's a case where the abilities of HEVC become very visible. I'm not sure though, whether 4k makes sense at all when you have just 1Mbps available.
It's a combination that Emby doesn't even use or allow to configure.

We take user issues serious, so please create a new topic, so we can take a look at it.
Let's not discuss this here, but two notes up-front: when there's buffering, it's usually either the server not transcoding fast enough or the target bandwidth isn't met or not calculated correctly. And with regards to quality: the default H.264 transcoding settings are set for moderate quality, it's not at the edge of what's achievable with H264.

There's no issue, I wasn't trying to sound critical, my limited bandwidth is the sole reason. If I had 1gig up/down, I wouldn't really care too much. For me personally, unless I'm out of the house, I'm on the LAN and emby works great. I actually run jellyfin and emby in tandem because I've been testing performance, quality, and just overall evaluating the two. But most of my family/friends, who are 100% WAN, have organically moved to using the Jellyfin apps, due to the quality increases, which in some cases, are quite dramatic.

I encode 4k content whenever possible in HEVC or AV1. That's just a default for me when storing the source files. Obviously, the guys on the WAN won't be able to watch it in 4k quality, but it's much clearer when using HEVC transcoding, no discernible pixelation on a 4K TV, it looks quite nice.

The only thing I would like to achieve by posting here, is encourage the emby team to consider prioritizing high efficiency codecs to some degree. I know it's not necessarily easy, but if your target audience is home users, high efficiency codecs are going to provide that user base a better experience. Unless most people have amazing upload speeds and I truely am an edge case. I appreciate your responses, they've been very detailed. 

[japtain_cack 6]

33 minutes ago, softworkz said:

HEVC has benefits - there's no doubt about that. It's all about outweighing the benefits vs. the required effort.

Sure. That's a case where the abilities of HEVC become very visible. I'm not sure though, whether 4k makes sense at all when you have just 1Mbps available.
It's a combination that Emby doesn't even use or allow to configure.

We take user issues serious, so please create a new topic, so we can take a look at it.
Let's not discuss this here, but two notes up-front: when there's buffering, it's usually either the server not transcoding fast enough or the target bandwidth isn't met or not calculated correctly. And with regards to quality: the default H.264 transcoding settings are set for moderate quality, it's not at the edge of what's achievable with H264.

Also, I have set my h264 CRF to 20 and slow, and tuned for quality as much as I can within emby, which should produce fairly high quality content for h264. I've tried changing that up and setting it to crazy levels, but even so, h264 is still blurry/pixelated at the 1Mbps setting. I'm not sure if the moderate h264 transcoding settings you mentioned are the ones I changed or something embedded, but at these bandwidth ranges, it's still fairly low quality compared to hevc.

[softworkz 5066]

1 minute ago, japtain_cack said:

Also, I have set my h264 CRF to 20 and slow, and tuned for quality as much as I can within emby, which should produce fairly high quality content for h264. I've tried changing that up and setting it to crazy levels, but even so, h264 is still blurry/pixelated at the 1Mbps setting. I'm not sure if the moderate h264 transcoding settings you mentioned are the ones I changed or something embedded, but at these bandwidth ranges, it's still fairly low quality compared to hevc.

Are you using hw acceleration? Those values apply to software H.264 encoding only.

[japtain_cack 6]

Just now, softworkz said:

Are you using hw acceleration? Those values apply to software H.264 encoding only.

Ahh, that would explain why changing them seemingly has no effect haha. 

[softworkz 5066]

Just now, japtain_cack said:

Ahh, that would explain why changing them seemingly has no effect haha. 

I know. It's awfully misleading. Please install the Diagnostics plugin. It will show you parameters for the hw encoders on the "Advanced Transcoding" page.
Embarrassingly, these settings don't survive reboots (yet).

[japtain_cack 6]

1 hour ago, softworkz said:

I know. It's awfully misleading. Please install the Diagnostics plugin. It will show you parameters for the hw encoders on the "Advanced Transcoding" page.
Embarrassingly, these settings don't survive reboots (yet).

I'll give that a shot. My server running emby is quite beefy, so I'm sure I can bump stuff up a bit without issue. 

Edited October 29, 2022 by japtain_cack

[Carlo 4560]

8 hours ago, syadnom said:

'same quality' is subjective.  You can use MOS scores by consuming the content but IMO, that's not a clear indicator either.  It's easy to refer off to major players who have spent the money to do market research though.  

should also be noted that most modern security cameras are doing on the fly h.265 encoding and with hardware encoding very similar to quicksync or nvenc.    It's a good test case for quality vs bitrate if you force some settings (fixed GOP and key frames etc)
 

No, there is software you can run that analyzes the results of an encode frame by frame back to the original frame that tells you specifically ie 85.46% the same. The software can also tell you how fluid it is as well as other factors comparing the encode to the source.  So there doesn't need to be anything subjective about it. I didn't get into details but when I said you could adjust the HEVC bitrate/parameters to get the same quality output this is what I was referring to.

Unfortunately, a lot of what you read in forums is antidotal information based on the person who did the encode and is very subjective as you say! They could easily be off 10% or more and think one encode is better than the other because they think it should be because of their expectation based on the setting they used or based on the outcome they expect to get.  Had they asked at least two or three other people (but not nearly enough) to view two streams and tell you which seems better and why without saying anything else they may have gotten a surprise result or heard why the person didn't like one compared to another as the person doing the encode didn't notice it.  As an example, pull down and de-interlacing that isn't 100% correct drives me nuts while a very small amount of banding in very dark scenes doesn't bother me that much but might drive another person batty because they want true dark scenes.  Perception of outcome is a hard thing to overcome for anyone as you typically will have a bias based on an expected outcome.  The way I semi got over that personally was writing a control script that encodes 15 to 30 second segments randomly adjusting a specific encoding setting between allowed values and then plays them randomly back, so the viewer has no idea what they are seeing.  You can even take the control (AVC) and have it randomly mixed in with the other encodes that you rank & score. Then it asks for a rank score you input for each.  It than allows playing back in the order you ranked them high to low or low to high to verify your choices are correct.  After that it shows you the results.  It of course is still subjective but if you didn't score the AVC version as the same as the control your other results are going to less than trustworthy. 

You take the results of this subjective test and then combine it with the analysis software that gives you specifics about the frame comparisons and you have a much better controlled experiment.

Actually, taking raw rips and doing the subjective test to find out what an acceptable loss of detail is that still looks very pleasing to you is a good skill to know. In software based encodes of H.264 you have a Constant Rate Factor you can use to tell the encoder what quality to try to achieve.  18 to 28 are typical values for DVD with 23 being the default. The combination for CPU based H.264 encoding using the x264 encoder in ffmpeg uses a combination of Preset and CRF to achieve a specific quality and size (all else being equal).  You can then try lowering or raising the preset (veryslow, slower, slow, medium, fast, faster, veryfast, superfast) and adjusting the CRF to change the speed of the encode while trying to keep your quality reference.  The results you find will likely be using slower or slow is preferred for offline conversion but anything from medium to veryfast is needed for real-time transcoding based on number of simultaneous transcodes as welll as performance of your CPU(s) in the machine. You might find you want a lower CRF (ie 20) for storage but can run up to 25 for real-time encoding and be happy with the results.

Using the subjective and software analysis together will give you results that should be pretty accurate.  You can then do the same for QuickSync and/or Nvidia AVC transcoding for both long term storage and real-time transcoding.  Same with CPU or GPU based HEVC transcoding.

My point of this post is that you need actual methods to analyze results otherwise your own bias and expected outcome will negatively influence your results.  But following some methodology like this you can get pretty accurate results and then be able to see what kind of bitrate savings one profile or codec gives compared to another.  You could for example re-examine what your current settings are for H.264/AVC and tune them to be more ideal for both conversions and transcodes.  You need to do this with different media to find what works for all types and gives the best overall results and set that for real-time transcoding.  For conversions you could actually use different settings for animation, different for average drama or Comedy films while using yet a different profile for action films. These could even use 2 passes encoding to make the best use of the bitrate you're targeting. For offline conversions you can play around to really fine tune and have many profiles available to use.  Media with an intentional film grain look is going to be bitrate heavy unless you take that into consideration.  Offline conversions offer a lot of freedom to encode samples and tune per movie or batch of like media that isn't available to you for real-time encoding in general use.

So getting back to real-time. You can compare using analysis software the outcome of an AVC encode to the original as well as an HEVC encode to the original.  You can then compare the time and latency each would add in use compared to a direct stream.  Now you compare the bitrate needed for AVC and the bitrate needed for HEVC and compare them.  If you're real-time transcode is close to the same using a GPU or CPU because you're trading resources for time and won't be able to handle as many streams (which may be fine) but you have a significant saving in bitrate you have a winner.  If you find your HEVC takes X amount longer and has for example 20% bitrate savings, does the fact you can transfer this through the internet save you latency compared to AVC and if so was it more than the latency you added? If close it's probably TIE-WIN as it will help with mobile data and data caps. If you're using a high end XEON or EPYC system you could throw a lot more CPU cores at the transcoder to reduce latency transcoding. The HEVC encoder scales cores much better than h.264 does due to the nature of the compression. In this case you would have saved overall latency because you'll have 30% to 55% savings in bandwidth depending on media transcoded which also means a 30% to 55% savings in latency for the stream transfer. This is a WIN in every checkbox except resource usage.

You can measure the quality of the encoded videos with SSIM, PSNR, or the popular vmaf from Netflix. ffmpeg has a switch to support vmaf quality analysis.

PS for anyone currently stuck with a small upload pipe using a GPU for transcoding you should test CPU only transcoding, if possible using 25 CRF and veryfast and see what you're speed is like as well as the quality of the transcode.  Next try CRF 23 (default). CRF should be your guide for quality needed while the preset will determine how fast it can run to achieve that setting.  very fast will work a lot faster then other preset but generate larger files but still probably smaller than GPU. A preset of Normal and CRF of 23 should generate pretty ideal bitrates.  The higher the CRF you can tolerate combined with more efficient processing (slowest preset that works) will generate the smallest possible streams. That could be a better solution that some of you can use that were looking to HEVC for bitrate savings. 

[softworkz 5066]

@cayars - Nice writeup!

Just a little annotation:

1 hour ago, cayars said:

The HEVC encoder scales cores much better than h.264 does due to the nature of the compression.

The idea makes sense and it applies to the case when only frame-threading would be used. Both encoders (x264 and x265) support slice-threading, though, which is active by default. Slice threading means that a single frame is divided into multiple slices each which is processed by a separate thread.
x264 creates one thread per core by default which is the reason why you often see your CPU near 100% even with decent hardware. This can be avoided by using the TranscodingThreadCount option to reduce the amount of threads being created (plus activation of throttling).

[Carlo 4560]

On 10/28/2022 at 3:16 PM, japtain_cack said:

I would also like to point out something quite obvious, that many people seem to overlook. Emby/Plex/Jellyfin have one function that sets them apart from other apps like Kodi. Real time transcoding/streaming. This is essentially the only thing that makes them unique.

If I wanted to re-encode multiple versions of the same file, not only am I not benefiting from the space savings of HEVC/AV1, but I'm essentially removing the need for real time transcoding, so why do I even need emby at that point? I could just use Kodi. Kodi supports all these codecs, organizes my library, plays pretty much any media, and looks great. 

The entire reason people use emby, or the like, is for real time transcoding. So, imho, if transcoding is your niche, you should do it well, and try to be the best at it. Whoever does it best, emby vs jellyfin vs plex, is the one that will dominate the market.

I don't pay for mainstream streaming services, but instead pay/donate monthly to jellyfin, since they are the clear winner when it comes to transcoding. If emby wants to start doing it better, I may change my mind.

I highly encourage everyone to donate to the free/OSS software they consume, if possible.

It's not really the real-time transcoding that sets them apart as lots of people's servers never have to transcode much of anything (prepaired media), but it's the fact it's a full eco-system media setup.  It has both a multi-user streaming server as well as clients that basically run on all the popular devices. It has a clean TV and DVR built in that doesn't require wild kodi setups to use and it doesn't need shared or exposed storage.

Kodi for sure has die hard fans but a lot of people have switch to full blown media servers like Emby or to very simple GUIs like TiviMate and feed it content via m3u files which could point local or to streaming services.  This type of solution is very easy to keep synced or updated as you can have it pull the m3u file from a cloud provider.

Kodi has very little of this built in as it's just a media client that has a nice scripting environment available that people use to access info on other servers. Besides that, Kodi can get unruly trying to manage a large collection as it wasn't built to do that. 

Some people may be drawn to Emby for the transcoder so most media is playable on any device.  But just because they switched to have that feature doesn't mean they want to always have to use it.  You can learn to preprocess your media before adding it to Emby so it normally will never get transcoded. So that's one offline transcode/conversion vs every time the media is played. Maybe only preprocess what you know will be popular and played multiple time instead of creating a backlog. Or alternately files over X size (your largest) get processed first to save space.  What ever, works as there is the real-time transcoder available when needed. But keep in mind anything that can direct play and has been properly processed offline is going to have a lot lower latency during playback.

As far as the re-encode goes, I've been running HEVC for a few years and made it my default.  But I also created AAC default 2 channel tracks if not present to stop transcodes & remuxes from happening due to lack of audio support.  I also house cleaned the files by, stripping all audio and subtitles not in English as my family/friends have no use for this. I then exported subrips to external SRT if present and then finally converted the video stream to h.265 then created a new package for the movie in mkv format.  What I have now are tiny streams compared to the originals and 1/3 to 1/2 the size they would get transcoded by GPU to in real time even using h.265. They still look great on my 75" 85" and 110' TV.  The streams direct play basically all the time unless someone is on mobile with a local limit set.  I got big space savings from doing this.

 

JellyFin is pretty far behind Emby and Plex when it comes to hardware transcoding.  Don't know how you think they are the clear winner.  They use much of our code as well as published Plex code whenever possible to make improvements but it's the slowest of the bunch especially for modern files that are 4K HDR.  On my 920+ Synology NAS for example I can't get a 4K HDR to 1080p SDR transcode from JF, can get 1 stream from Plex and 2 or 3 from Emby depending on the bitrate but there is clearly a difference. They are missing a lot of functionality and have issues to work out for things like subtitle.  Their transcoder still feels like a 4 year older version of Emby''s (which it was/is).  They add stuff but don't really understand how it interacts with other parts of the system.  They recently tried to add some h.265 code but had to revert it back out as they didn't realize it's not a drop in code.

 

Edited November 2, 2022 by cayars

[mediaGuy 8]

I stream Emby (HDHomerun) to a cottage which can't receive TV signals and it's internet is ~ 3Mbps.  So for me, switching to HEVC would help quite a bit.

I understand there are complications, but @softworkz discussion in the last few days was really the first insight on the status of this feature request.  If the emby team doesn't want to include HEVC then just close the thread with a statement to avoid people waiting and guessing.

As mentioned before, I used HEVC on tvheadend for years with no issues, streaming to Kodi on Windows, AndroidTV and linux.

[softworkz 5066]

9 minutes ago, mediaGuy said:

 If the emby team doesn't want to include HEVC then just close the thread with a statement to avoid people waiting and guessing

The one thing that this statement reveals is that you either haven't carefully read all of my posts or you didn't understand them or you are pretending the latter.
This is so boring. Now I have to clarify another time:

What I have explained is the current situation right now, in October 2022 - and that's all. Technology is evolving, standards are changing and new standard coming up. Industry adoption of standards is evolving as well and sometimes very quickly especially this subject is something that needs frequent re-evaluation.

I also want to spell it out once again for all those who are just begging for something that they could pretend to misunderstand:

There is no strategic avoidance towards H.265 at all. I didn't say no and I didn't say never. In fact there are planned projects related to H.265, which are making use of that codec within a defined and limited scope.

What I said in those posts above though, was about introducing HEVC transcoding as a general replacement to the H.264 streaming that Emby is doing. This is what this feature request is about and I have explained why this is not as easy as one would think.

[VirulentPip 93]

"What I said in those posts above though, was about introducing HEVC transcoding as a general replacement to the H.264 streaming that Emby is doing. This is what this feature request is about and I have explained why this is not as easy as one would think."


OP Request -  "I would like to have an option to choose H265 instead of H264 in the Transcode option."

Sorry to be that guy.. But the OP asked for it to be an option to choose H265.. Not to replace it 

I can see the reasons against replacing, that would be bad. But having the option to choose which to transcode to would be great. 

[ebr 16169]

1 hour ago, mediaGuy said:

it's internet is ~ 3Mbps.  So for me, switching to HEVC would help quite a bit

Hi   I think you are making an assumption there.  Transcoding to HEVC on the fly may or may not make a big difference for you but, in that situation, pre-encoding lower bitrate (1-2Mb I would think) versions of your files definitely would make a big difference.

[ebr 16169]

2 minutes ago, VirulentPip said:

Sorry to be that guy.. But the OP asked for it to be an option to choose H265.. Not to replace it

Hi.  But, if one chose that option, that would replace all transcodes with HEVC instead of AVC.  So, functionally, it would be the same once someone threw that switch.

[softworkz 5066]

14 minutes ago, VirulentPip said:

"What I said in those posts above though, was about introducing HEVC transcoding as a general replacement to the H.264 streaming that Emby is doing. This is what this feature request is about and I have explained why this is not as easy as one would think."

OP Request -  "I would like to have an option to choose H265 instead of H264 in the Transcode option."

Sorry to be that guy.. But the OP asked for it to be an option to choose H265.. Not to replace it 

I'm sorry for the misunderstanding!

What I meant by "general replacement" is not replacing it in a fixed/hard way. By replacement, I meant that it could be (optionally of course) used in-place-of and in the same way and with the same coverage of cases like H.264.

Edited October 29, 2022 by softworkz

[VirulentPip 93]

6 minutes ago, ebr said:

Hi.  But, if one chose that option, that would replace all transcodes with HEVC instead of AVC.  So, functionally, it would be the same once someone threw that switch.

Maybe it could be at user level X User - H265 transcodes, Y User - H264 transcodes. I dunno how that would be to implement, I'm just spit balling. You lot obviously have the knowledge, I'm just a newb haha. 

[softworkz 5066]

If you want to ask a good question, you would ask me, whether H.265 couldn't be enabled only for those cases where it's working.

[ebr 16169]

8 minutes ago, VirulentPip said:

Maybe it could be at user level X User - H265 transcodes, Y User - H264 transcodes. I dunno how that would be to implement, I'm just spit balling. You lot obviously have the knowledge, I'm just a newb haha. 

Right and then that gets into the complexity of the actual implementation and making this work for everyone - as detailed by Softworkz above.

The bottom line as of this point and time:

1. The benefits people think they will get from this feature are not quite as high as most people think
2. The cost to properly implementing this feature at this point in time far outweigh those potential benefits
3. In the future, there very well may be a better solution to the real problem this request is attempting to address (limited bandwidth delivery of high-quality content).

This is a great example of a situation where it is really best to file a feature request at a functional level instead of for a particular detailed implementation.

This request was "give us the option to transcode to h265" but what is really desired is "improve quality and speed of media delivery in limited bandwidth environments".