Transcode in H265

[Sshanee 3]

6 years later still no 265 or av1 live transcoding options. Just softworks acting like a douchebag and Carlos making long winded excuses. This would benefit SO MANY people and nearly ANY and ALL WAN streaming use cases.

How about instead of spending so much effort arguing with your users, you put a little effort into giving them what they're asking you for? If you offered this, nobody would have any reason to use jellyfin. Why is this so hard to comprehend for you?

I'm becoming irritated by this team's attitude and lack of respect for the community. I am considering moving to jellyfin due to this factor alone, forget thier superior streaming capabilities and the fact they are listening to their users...

Softworks in particular is creating a very bad image for Emby and should be banned from using the forum all together. It's like reading trumps tweets.

Edited September 5, 2023 by Sshanee

[speechles 1918]

@CarloHow old was that chart you posted earlier in this thread? It is showing wrong information for Roku.

 

This is the correct information for the Roku direct from Roku.co themselves.

[softworkz 3335]

2 hours ago, Sshanee said:

Just softworks acting like a douchebag and Carlos making long winded excuses.

There's nothing we need to excuse for.

 

2 hours ago, Sshanee said:

How about instead of spending so much effort arguing with your users, you put a little effort into giving them what they're asking you for?

My standpoint is that it's more beneficial for both, users and our side, to explain the background, challenges and reasons and provide a realistic picture of what can be expected and what not. But perhaps I'm wrong and it's better to do like usual: saying "yes, it's possible for the future" and repeat the same each time someone is asking about it.

In this regard, i can only say: "yes, it's possible for the future". 

3 hours ago, Sshanee said:

If you offered this, nobody would have any reason to use jellyfin. Why is this so hard to comprehend for you?

And why Isn't everybody using JF? 

Because they can throw out some experimental features which are working fine sometimes and other times not. They don't owe anything to users. They can say "We have XY working" when it's working in a few cases only. We/Emby can only say that when something is working always (when applicable).

That is a huge difference. 

Why is this so hard to comprehend for you?

[Sshanee 3]

1 hour ago, softworkz said:

There's nothing we need to excuse for.

 

My standpoint is that it's more beneficial for both, users and our side, to explain the background, challenges and reasons and provide a realistic picture of what can be expected and what not. But perhaps I'm wrong and it's better to do like usual: saying "yes, it's possible for the future" and repeat the same each time someone is asking about it.

In this regard, i can only say: "yes, it's possible for the future". 

And why Isn't everybody using JF? 

Because they can throw out some experimental features which are working fine sometimes and other times not. They don't owe anything to users. They can say "We have XY working" when it's working in a few cases only. We/Emby can only say that when something is working always (when applicable).

That is a huge difference. 

Why is this so hard to comprehend for you?

What is your role at Emby? Are you the one who would write the code to implement such a feature? If not, could you kindly be quiet and let someone who has the knowledge and coding capacity for this feature speak? It's okay that you don't know how, but we shouldn't speak for others.

[Luke 37068]

1 minute ago, Sshanee said:

What is your role at Emby? Are you the one who would write the code to implement such a feature? If not, could you kindly be quiet and let someone who has the knowledge and coding capacity for this feature speak? It's okay that you don't know how, but we shouldn't speak for others.

He would be involved in it, yes. Since we have the feature for conversion, we'll take a look at how feasible this might be to make as an option and label as experimental. 

[softworkz 3335]

1 hour ago, Sshanee said:

What is your role at Emby?

I'm  the facility manager, unlocking and locking the rooms each mornings and evenings and bringing out the trash.

[Sshanee 3]

1 hour ago, Luke said:

He would be involved in it, yes. Since we have the feature for conversion, we'll take a look at how feasible this might be to make as an option and label as experimental. 

I understand why there was hesitation with 265 but with av1 already here and modern gpu and cpus able to encode it, its probably time to look at this more seriously

[Carlo 4330]

On 9/5/2023 at 1:31 PM, speechles said:

@CarloHow old was that chart you posted earlier in this thread? It is showing wrong information for Roku.

 

This is the correct information for the Roku direct from Roku.co themselves.

I don't see anything conflicting as there is no mention of AV1 above which is what my chart was showing support for. That chart is for AV1 only and not AVC, HEVC, VP9, etc

Source was about 3.5 months old at the time I posted it. It's from an articled dated 11/27/2022.  https://www.gtc.pt/more/blog/the-state-of-av1-playback-support-2022/

[speechles 1918]

@CarloAs stated it is useless to compare to today. Thanks for confirming.

Also.. have you read the topic of this thread? or anything tagged to this thread? Where is AV1 mentioned? The original topic of this thread was already lost long ago. This thread should've been split or adjusted when others chimed in with different codecs. As it doesn't make sense to mention AV1 when the OP wasn't interested in that. Now it is just weird.

Edited September 7, 2023 by speechles

[ebr 14913]

38 minutes ago, speechles said:

Where is AV1 mentioned?

Many times it has come up in this thread as a possible alternative to HEVC.  However, it has its own set of challenges as well.

38 minutes ago, speechles said:

As it doesn't make sense to mention AV1 when the OP wasn't interested in that

This topic should have actually been labeled "Transcode to a more bandwidth efficient codec" because that is really what it is discussing but people often just see what they think is a solution to their problem and ask for that specific implementation.  We should always be looking for the higher level problem and find the best solution to that.

[softworkz 3335]

Regarding AV1 Encoding

AV1 encoding in > realtime speed is going beyond the limits of even the latest CPU models unless there's significant downscaling before encoding.
Less than 0.1% of Emby users have hardware available which supports AV1 hw encoding 
Emby uses HLS for streaming and I'm not aware of any client player which is explicitly supporting AV1 on HLS.
Maybe the "eat-anything" MPV player can do it, but even when it would, it would still be < 10% of clients.

Doing the Math: 0.1% of servers times 10% of clients makes: 0.01% of cases: If we would spend effort on developing this now, only one out of 10,000 playback situations would benefit from this work.

In a few years, the situation will possibly be very different. But for now - AV1 encoding is totally off.

[sshanee613 3]

Netflix has been streaming av1 to fire TV devices since 2021. Intel arc gpus, nvidia 4000 series and amd 7000 series gpus all encode av1 as well as all the latest generation cpus. They are more than capable of handling the task. Every new device should be compatible with av1. The Fire TV 4k max being one of them and was released in 2018.

 

I'm not knowledgeable about HLS particulars but if it isnt future compatible then maybe it's time to swap it out for a protocol that is. Av1 is the future, its not speculation. It's already here devices can already play and encode it. Why not get ahead of the curve this time around instead of 6 years behind it?

 

That's all you'll hear from me about this topic. I hope emby will make the right decision this time around

[ebr 14913]

11 hours ago, sshanee613 said:

Netflix has been streaming av1 to fire TV devices since 2021

Hi.  That is pre-encoded content.  Completely different situation.  You can do the same now by pre-encoding your content.

[jaycedk 382]

I pre-encode all my files into HEVC using Tdarr.

Drop media into tdarr watch folder, and it does it for you.

TV-show in 1 folder, movies in another etc.

And when done, its placed into Emby library respectively.

 

[spencerjford 1]

How about making it is available to encode to hevc or av1 IF the system detects a GPU doing the encoding, and unavailable if doing CPU encoding?  Seems like a decent compromise to the use case, or am I way of on this thought process?

 

[softworkz 3335]

22 hours ago, sshanee613 said:

Intel arc gpus, nvidia 4000 series and amd 7000 series gpus all encode av1 as well as all the latest generation cpus

That's correct. What's also correct is that < 0.1% Emby users have such hardware (at the time of writing).

22 hours ago, sshanee613 said:

I hope emby will make the right decision this time around

There's no doubt that we will have to move on and leverage newer technologies at some point. I had written all these things earlier in this conversation, but I understand that it's easier to attack Emby or Emby staff instead of taking the time to carefully read through messages and trying to understand the intentions.

 One of the core points in this context is exactly about "making the right decisions". Emby is still quite small and we cannot afford going into multiple directions in parallel and throwing away the less successful ones later, which companies like Netflix can do with ease.

Like also explained earlier, there isn't a clear "winner" strategy visible on the horizon for which we an say "that's it - let's go all-in for this strategy". If that would be the case, then we would have already started working towards it.
After the JF fork, we had fundamentally reworked our transcoding implementation and the differences in quality and reliability you are seeing, are a direct result of this - but it has been a tremendous effort over several years and when investing a similar effort again for a newer streaming/encoding technology, we need to be sure that it's "the right decision/target".

Also explained above: Delivering some "experimental" implementation with something like HEVC encoding would be a piece of cake. Nobody needs to doubt that we couldn't do the same like JF does, but there's a difference in expectations: When JF offers something which is working in a fraction of cases but failing in all others, then people are applauding and coming to tell "JF can do XY".  But they are telling it to us with the expectation that when we would provide the same feature, that it must  be "always working" and not just sometimes.

Users wouldn't applaud us for providing a feature labeled "experimental" which isn't better than JF's implementation. And right they are: We are Emby and users can expect more from us than a partially working experimental implementation - because "partially working" implies "frequently failing". 

We aim to provide best-in-class features and implementations for Emby users just like we've been doing over the past years already:

• We have the most flexible and adaptive transcoding pipeline creation, based on detailed detection of transcoding hardware availability and capabilities, like no other in the competition
• We had D3D11 based QuickSync transcoding on Windows, two years before it was added to ffmpeg
  Eventually, Intel had abandoned their own implementation (submitted to ffmpeg) in favor of mine (for which I had given them permission to submit it as 'Intel')
• We have the fastest software tone mapping implementation for x86 architectures, using SIMD/vectorization
• We have the fastest OpenCL tone mapping

And we are dedicated to continuing on that path.

But providing the best possible experience, can sometimes also mean to be hesitant in cases where it's clear that there's nothing to win and possibly even a lot to lose.  

At least as far as I'm concerned, it's one of the worst things for a personal media server, when you want to enjoy your media and start playback, but it ends up with an error. Who likes to stand up in such situation and even possibly be required to tell others people in the room something like "Wait, I need to tweak some settings first!"

Edited September 9, 2023 by softworkz

[softworkz 3335]

7 hours ago, spencerjford said:

How about making it is available to encode to hevc or av1 IF the system detects a GPU doing the encoding, and unavailable if doing CPU encoding?  Seems like a decent compromise to the use case, or am I way of on this thought process?

Unfortunately, It's a hundred times more complicated than that. You may want to re-read my earlier posts in this conversation where I've tried to give some closer insights into the subject. 

Thanks.

[sebasmiles 8]

On 9/8/2023 at 11:20 AM, jaycedk said:

I pre-encode all my files into HEVC using Tdarr.

Drop media into tdarr watch folder, and it does it for you.

TV-show in 1 folder, movies in another etc.

And when done, its placed into Emby library respectively.

 

Seems this is the best path for now I guess, is it fairly good in direct streaming h265 when its capable? did you do something specific with the subs to avoid having it transcode?

Any guide you used would be helpful, I saw someone else try it and it was hit or miss (saw some quality degradation)

[softworkz 3335]

Pre-encoding files for Emby is actually a bad idea in general. It can make sense in special cases, like when you know for sure that your playback situations are always Direct Play, no downscaling, no tone mapping, no deinterlacing, no subtitle burn-in will be required. In other words: when you are sure you never need any transcoding.

But those are rare cases. For all other cases, the recommendation is: 

1. Do nothing to your media, leave it all as is and let Emby dynamically decide what to do on a case-by-case basis. 
   Nobody can make these case-specific decisions better than Emby  (even I for myself couldn't generally do it better than Emby because there's so much experience which has flown into all the decision logic, and client-server interaction that a single person couldn't remember it all at once
2. Very rare cases exist, where a re-encoding can be helpful to work around known issues with a certain format or encoding.
3. Re-Encoding from H.264 to H.265/HEVC is a rather bad idea, because it eliminates the chance of re-muxing (transcoding where just the audio is re-encoded or nothing is re-encoded and the media streams are just filtered and segmented (for HLS streaming) 
   It's also a bad idea, to do it for saving disk space, because when encoding to HEVC with settings to save space, you will end up having a sparse keyframe rate, which is not suitable for streaming and which increases delays when skipping video to specific positions.
   Also, re-encoding to HEVC doesn't mean that you will get HEVC streaming (with a single specific experimental exception). It only affects DirectPlay situations.

I know that some will respond now, saying how well it works for them, but it's in no way comparable to extracting a zip and re-compressing as 7z or similar. We've seen so many playback issues in the past due to incorrect manual re-encoding of videos, that I can only recommend to think twice. 
Even for myself - as  someone who would probably be able to do it right - I would never consider re-encoding my media files for space savings or other reasons - and in neither direction (from/to codec) - in summary, the disadvantages will almost always outweigh the advantages.

[Carlo 4330]

On 9/8/2023 at 9:41 PM, softworkz said:

That's correct. What's also correct is that < 0.1% Emby users have such hardware (at the time of writing).

If we build it, they will come.   I think we need to treat codecs beyond H.264 differently and stop thinking about CPU use for real-time encoding. The codecs will be more and more complicated and CPU/GPU intense with more pixels to process, as screen sizes and fps get larger and larger.

I'm not sure the number is < 0.1% but I bet if we had a solution using NVidia 4000 series, Intel ARC or other dedicated hardware the percentage would rise accordingly. Right now there is probably little reason for anyone to have this hardware if it's only used for Emby.

On 9/9/2023 at 6:16 PM, softworkz said:

Pre-encoding files for Emby is actually a bad idea in general. It can make sense in special cases, like when you know for sure that your playback situations are always Direct Play, no downscaling, no tone mapping, no deinterlacing, no subtitle burn-in will be required. In other words: when you are sure you never need any transcoding.

But those are rare cases. For all other cases, the recommendation is: 

1. Do nothing to your media, leave it all as is and let Emby dynamically decide what to do on a case-by-case basis. 
   Nobody can make these case-specific decisions better than Emby  (even I for myself couldn't generally do it better than Emby because there's so much experience which has flown into all the decision logic, and client-server interaction that a single person couldn't remember it all at once
2. Very rare cases exist, where a re-encoding can be helpful to work around known issues with a certain format or encoding.
3. Re-Encoding from H.264 to H.265/HEVC is a rather bad idea, because it eliminates the chance of re-muxing (transcoding where just the audio is re-encoded or nothing is re-encoded and the media streams are just filtered and segmented (for HLS streaming) 
   It's also a bad idea, to do it for saving disk space, because when encoding to HEVC with settings to save space, you will end up having a sparse keyframe rate, which is not suitable for streaming and which increases delays when skipping video to specific positions.
   Also, re-encoding to HEVC doesn't mean that you will get HEVC streaming (with a single specific experimental exception). It only affects DirectPlay situations.

I know that some will respond now, saying how well it works for them, but it's in no way comparable to extracting a zip and re-compressing as 7z or similar. We've seen so many playback issues in the past due to incorrect manual re-encoding of videos, that I can only recommend to think twice. 
Even for myself - as  someone who would probably be able to do it right - I would never consider re-encoding my media files for space savings or other reasons - and in neither direction (from/to codec) - in summary, the disadvantages will almost always outweigh the advantages.

We have different thoughts on 1,2 & 3 above. I've always been an advocate for doing a high quality encode to a standardized format used in my library. I've got QA checking built into my pipeline that checks the quality against the source to make sure I don't degrade more than x% and make sure I've saved at least 30% diskspace otherwise I don't convert the file. I may be able to instead remux the file making some corrective actions such as adding a 2 channel audio track if needed along with corrections to the subs if needed.

Deinterlacing can be done the way you prefer this way. This allows for direct play using the least amount of bitrate possible. So less disk space needed to store the files as well as lower latency and less bandwidth needed to stream the files. If transcoding needs to be done, then I know the files are in a format it can easily handle. Done correctly the format used will allow remuxing as well to streaming formats.
Don't want to go down this path but I don't agree with #1, 2 or 3 above.

[sebasmiles 8]

40 minutes ago, Carlo said:

If we build it, they will come.   I think we need to treat codecs beyond H.264 differently and stop thinking about CPU use for real-time encoding. The codecs will be more and more complicated and CPU/GPU intense with more pixels to process, as screen sizes and fps get larger and larger.

I'm not sure the number is < 0.1% but I bet if we had a solution using NVidia 4000 series, Intel ARC or other dedicated hardware the percentage would rise accordingly. Right now there is probably little reason for anyone to have this hardware if it's only used for Emby.

We have different thoughts on 1,2 & 3 above. I've always been an advocate for doing a high quality encode to a standardized format used in my library. I've got QA checking built into my pipeline that checks the quality against the source to make sure I don't degrade more than x% and make sure I've saved at least 30% diskspace otherwise I don't convert the file. I may be able to instead remux the file making some corrective actions such as adding a 2 channel audio track if needed along with corrections to the subs if needed.

Deinterlacing can be done the way you prefer this way. This allows for direct play using the least amount of bitrate possible. So less disk space needed to store the files as well as lower latency and less bandwidth needed to stream the files. If transcoding needs to be done, then I know the files are in a format it can easily handle. Done correctly the format used will allow remuxing as well to streaming formats.
Don't want to go down this path but I don't agree with #1, 2 or 3 above.

I think worrying about AV1 can take us down the wrong rabbit hole, the odds people are spending that much on a GPU to get it should be very very rare but H265 is in an entirely different state as there are multiple generations of cards now with support for it.
I don't know enough of how the remux, deinterlace, subs works on the container of a 265 vs 264 if say it just needs to transcode the audio and how that impacts stuff if you set all the source to 265. So I defer to softworkz, you and others on it.

I do have a question about your process for re-encoding, are you using Tdarr? seems like you have quite a developed pipeline, any chance you could provide some info on how you have it set up?

[softworkz 3335]

1 hour ago, Carlo said:

If we build it, they will come.

They will come in 2-3 years when that hardware will have become mainstream rather than being at the top edge of the price range.

1 hour ago, Carlo said:

I think we need to treat codecs beyond H.264 differently and stop thinking about CPU use for real-time encoding.

Emby has always been very flexible and usable on a wide range of hardware from lowest to highest end. I don't see any reason to change that.

1 hour ago, Carlo said:

I'm not sure the number is < 0.1% but I bet if we had a solution using NVidia 4000 series, Intel ARC or other dedicated hardware the percentage would rise accordingly. Right now there is probably little reason for anyone to have this hardware if it's only used for Emby.

The latest Intel graphics do not even work with the 4.7 server, so you can put a zero behind this and only beta users are remaining. Nvidia's Ada Lovelace generation is still extremely high-priced, which doesn't stand in a healthy relation for using those "just" for an Emby Server - like you said. 
The vast majority of Emby Servers is running on mid-range to lower range hardware, which is clearly evident from looking at logs submitted alongside supported requests.

There's always a small number of enthusiasts who would buy such hardware in case - but well...small!

1 hour ago, Carlo said:

We have different thoughts on 1,2 & 3 above. I've always been an advocate for doing a high quality encode to a standardized format used in my library. I've got QA checking built into my pipeline that checks the quality against the source to make sure I don't degrade more than x% and make sure I've saved at least 30% diskspace otherwise I don't convert the file.

Everybody is free to do such things. My position is that the benefits are questionable and rarely justified, which you might not agree with, but you also need to consider that this is not a question of what YOU are doing - as somebody for whom these things are part of the professional work life.

It's about what normal Emby users should be doing. And they shouldn't do it:

• for once, because it's easy to do it wrong
• and secondly, because this is what they have Emby Server for them to do

Having Emby Server means that you don't need to care about those things - you give it whatever media you have and Emby plays it for you wherever you want.
You don't need to become a transcoding expert or spend endless hours on re-encoding your library. 

Doing these things for you - and in the right and best ways - is the job of Emby Server and an area which it handles with excellence.

Edited September 12, 2023 by softworkz

[softworkz 3335]

Clarification for partial readers:

There is absolutely no need to re-encode your media files for Emby Server

 

(some recent messages might have created a different impression)

[Dibbes 431]

8 hours ago, softworkz said:

Clarification for partial readers:

There is absolutely no need to re-encode your media files for Emby Server

 

(some recent messages might have created a different impression)

Not for Emby Server maybe, but for the wallet (storage, electricity, backups) re-encoding the majority of a library from H.264 to H.265/HEVC can have major advantages. Yes, it's easy to go wrong, but if you're willing to do the reading, there's tons of articles just on this forum which will give you a how-to or get a ready made script and there are always member of this community willing to help 

Edited September 12, 2023 by Dibbes

[softworkz 3335]

6 minutes ago, Dibbes said:

but for the wallet (storage, electricity, backups) re-encoding the majority of a library from H.264 to H.265/HEVC can have major advantages.

Storage - yes, but electricity - no.

When leaving files as H.264, there are chances that playback can be done by only remuxing the video (almost zero "electricity"). After re-encoding them to HEVC, they will always need to be transcoded - with two exceptions:

• DirectPlay
• An experimental and unreliable special playback case on some Apple devices.

Which brings us back to what I had written above:

On 9/10/2023 at 12:16 AM, softworkz said:

Pre-encoding files for Emby is actually a bad idea in general. It can make sense in special cases, like when you know for sure that your playback situations are always Direct Play, no downscaling, no tone mapping, no deinterlacing, no subtitle burn-in will be required. In other words: when you are sure you never need any transcoding.