Change hardware transcoding parameters?

[drapacioli 8]

Is there a way to adjust the hardware transcoding parameters in Emby? I have noticed that using Intel QuickSync leads to very noticeable blocking and artifacts when for example burning in subtitles. I'd like to be able to burn in subtitles with near-lossless quality, but it seems the transcoding is taking place at the same bitrate as the native file, so the quality is taking a hit. Can I specify it to use a higher bitrate, or a minimum bitrate/quality preset? All of our streaming is local or download, so the bandwidth isn't really a factor here, we just want to preserve the quality level where possible. The only settings I'm seeing relate to the software transcoding option. We tried the "allow subtitle extraction on the fly" but then the subtitles are ridiculously large and get cut off on screen in the player, and it doesn't respond to formatting/size adjustments, so the only real choice is to burn them in.

Edited September 11, 2024 by drapacioli

[drapacioli 8]

Example, here's a small screenshot of the browser playing back a video file with subtitles enabled and burning in, and disabled. The video itself is properly compressed with handbrake to preserve quality from the original while reducing filesize to something reasonable for long term storage. The result is a very clean image even with fast motion on screen and there's only minimal blocking and artifacts (on the right). But when the transcoding takes place, it uses the same highly compressed source bitrate value, and that results in a blocky mess.

 

 

I think if we were able to set a minimum transcoding bitrate to, say, 6-8mbps instead, that would result in a much better experience without taking up much in terms of extra resources, but I can't figure out a way to specify this even in the advanced hardware encoding options. I can only pick a preset like show below, but none of them allow for advanced encoding options or specifying the perceived "bitrate" or "maxbitrate" values that seem to be used internally:

 

I can change the encoding preset, but even putting that down to something insane like slow doesn't seem to do much in terms of the end quality level, very minimal difference from what I've tested so far. I'm not sure where else I can look?

[Luke 42077]

Quote

I can change the encoding preset, but even putting that down to something insane like slow doesn't seem to do much in terms of the end quality level, very minimal difference from what I've tested so far. I'm not sure where else I can look?

Hi, did you verify the changes were applied by comparing the ffmpeg logs?

[drapacioli 8]

12 hours ago, Luke said:

Hi, did you verify the changes were applied by comparing the ffmpeg logs?

I guess? If I'm looking in the right place in the logs, it looks like it:

File with default transcoding: 

C:\Users\nick\AppData\Roaming\Emby-Server\system\ffmpeg.exe -loglevel +timing -y -print_graphs_file "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\logs\ffmpeg-transcode-43393baa-1c3e-4c60-b315-289f3f73bafe_1graph.txt" -copyts -start_at_zero -init_hw_device "qsv=dev1:hw,child_device=0,qsv_use_dx11=1" -f matroska,webm -ss 00:00:06.000 -c:v:0 h264_qsv -threads:v:0 1 -hwaccel:v:0 qsv -hwaccel_device:v:0 dev1 -hwaccel_output_format:v:0 qsv -noautorotate -i "\\DS218\Emby\Anime\Subbed\Pocket Monsters (2023) {anidb-17777}\Pocket Monsters (2023) - E63 - Episode 63.mkv" -filter_complex "[0:0]hwdownload@f1,format@f2=pix_fmts=nv12,format@f3=pix_fmts=yuv420p,subtitles@f4=filename='\\\\DS218\\Emby\\Anime\\Subbed\\Pocket Monsters (2023) {anidb-17777}\\Pocket Monsters (2023) - E63 - Episode 63.mkv':fontsdir='C\:\\Users\\nick\\AppData\\Roaming\\Emby-Server\\programdata\\fonts':si=0,hwupload@f5=extra_hw_frames=32[f5_out0]" -map [f5_out0] -map 0:1 -sn -c:v:0 h264_qsv -b:v:0 3458005 -g:v:0 72 -maxrate:v:0 3458005 -bufsize:v:0 6916010 -keyint_min:v:0 72 -r:v:0 23.976024627685547 -profile:v:0 high -aud:v:0 1 -c:a:0 libmp3lame -ab:a:0 192000 -ac:a:0 2 -metadata:s:a:0 language=jpn -disposition:a:0 default -max_delay 5000000 -avoid_negative_ts disabled -f segment -map_metadata -1 -map_chapters -1 -segment_format mpegts -segment_list "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\transcoding-temp\B3FAC3\B3FAC3.m3u8" -segment_list_type m3u8 -segment_time 00:00:03.000 -segment_start_number 2 -individual_header_trailer 0 -write_header_trailer 0 -segment_write_temp 1 "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\transcoding-temp\B3FAC3\B3FAC3_%d.ts"

Same file with advanced transcoding and preset set to slow:

 

C:\Users\nick\AppData\Roaming\Emby-Server\system\ffmpeg.exe -loglevel +timing -y -print_graphs_file "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\logs\ffmpeg-transcode-4ae161bb-3789-4a7b-8065-6b6840e2aba2_1graph.txt" -copyts -start_at_zero -init_hw_device "qsv=dev1:hw,child_device=0,qsv_use_dx11=1" -f matroska,webm -c:v:0 h264_qsv -threads:v:0 1 -hwaccel:v:0 qsv -hwaccel_device:v:0 dev1 -hwaccel_output_format:v:0 qsv -noautorotate -i "\\DS218\Emby\Anime\Subbed\Pocket Monsters (2023) {anidb-17777}\Pocket Monsters (2023) - E63 - Episode 63.mkv" -filter_complex "[0:0]hwdownload@f1,format@f2=pix_fmts=nv12,format@f3=pix_fmts=yuv420p,subtitles@f4=filename='\\\\DS218\\Emby\\Anime\\Subbed\\Pocket Monsters (2023) {anidb-17777}\\Pocket Monsters (2023) - E63 - Episode 63.mkv':fontsdir='C\:\\Users\\nick\\AppData\\Roaming\\Emby-Server\\programdata\\fonts':si=0,hwupload@f5=extra_hw_frames=32[f5_out0]" -map [f5_out0] -map 0:1 -sn -c:v:0 h264_qsv -b:v:0 3458005 -g:v:0 72 -maxrate:v:0 3458005 -bufsize:v:0 6916010 -keyint_min:v:0 72 -r:v:0 23.976024627685547 -preset:v:0 slow -profile:v:0 high -aud:v:0 1 -c:a:0 libmp3lame -ab:a:0 192000 -ac:a:0 2 -metadata:s:a:0 language=jpn -disposition:a:0 default -max_delay 5000000 -avoid_negative_ts disabled -f segment -map_metadata -1 -map_chapters -1 -segment_format mpegts -segment_list "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\transcoding-temp\1ACD02\1ACD02.m3u8" -segment_list_type m3u8 -segment_time 00:00:03.000 -segment_start_number 0 -individual_header_trailer 0 -write_header_trailer 0 -segment_write_temp 1 "C:\Users\nick\AppData\Roaming\Emby-Server\programdata\transcoding-temp\1ACD02\1ACD02_%d.ts"

I don't think that's the problem though, I think the problem is the bitrate is too low for realtime encoding profiles in the first place. How can I override it or better yet remove the bitrate limitation entirely/set by the user defined quality limit in the web player? I've noticed it's worse on some episodes than others, I'm thinking it's using the average bitrate of the file to set the "bitrate" in the encoding settings? The quality is consistent in the source from episode to episode, visually speaking, but the average bitrate is the only thing that varies, and that value seems to be what is passed along in the transcoding profile. It's the only reason I can think of that the transcoding can vary so much in terms of quality from episode to episode. 

Edited September 13, 2024 by drapacioli

[drapacioli 8]

So, I thought I asked about this before. Turns out I was right, I finally found my own thread on this from 4 years ago after much googling...would have been helpful if we could look up our own content I guess? I swear we were able to do that before.
 

Anyway, one of the tests we were apparently able to do back then was enforce a find/replace using a diagnostics plugin to override the max bitrate values. This (still) works, the problem is I'd have to go look up the bitrate value in the logs for every single file I want to encode at a higher resolution. I don't suppose this tool supports a wildcard for any bitrate value? That would just solve this problem straight up, I'd override it to something like 15mbps globally and be done with it. 

Edited September 13, 2024 by drapacioli

[Happy2Play 9780]

Not that I am aware of as it is specific value replacement. 

Better off changing the client playback quality to anything but Auto.

[drapacioli 8]

3 minutes ago, Happy2Play said:

Not that I am aware of as it is specific value replacement. 

Better off changing the client playback quality to anything but Auto.

Playback quality setting in the web player seemingly has zero bearing on the actual transcoding bitrate limit. It's set to something like 60mbps by default, it's transcoding at 3.5ish.

 

 

 

Edited September 14, 2024 by drapacioli

[Happy2Play 9780]

So you are talking about the remote client hardcoded 4Mbps Auto transcoding value, correct?

[drapacioli 8]

2 minutes ago, Happy2Play said:

So you are talking about the remote client hardcoded 4Mbps Auto transcoding value, correct?

I have no idea what it is referred to internally, but I guess? The behavior is if I play any video file that requires subtitle burn in, the transcoding is capped at the source file's average bitrate. This happens regardless of whether I'm viewing from localhost or another device on my network, or a device outside my network. The application restricts any transcoding to the source file bitrate even when it's impractical  in terms of preserving quality.

Edited September 14, 2024 by drapacioli

[Happy2Play 9780]

Never really agreed with artifically inflating bitrate on conversion but each there own I guess as bitrate is almost doubled trancoding h265 to h264.  But guess I have never seen an issue transcoding something at its native bitrate.

[drapacioli 8]

26 minutes ago, Happy2Play said:

Never really agreed with artifically inflating bitrate on conversion but each there own I guess as bitrate is almost doubled trancoding h265 to h264.  But guess I have never seen an issue transcoding something at its native bitrate.

Well I never really agreed with artificially limiting bitrate on conversion, that defeats the purpose of compressing a file to target a desired quality in the first place. I use optimized videos like this because I don't have thousands of dollars to throw at a NAS with capacity to store uncompressed/lightly compressed video, and I shouldn't have to. While the hardware encoders don't have much/any customization, The h.264 software encoding option page literally has a CRF value exposed that's supposed to let you preserve quality:

But this is not a CRF of 10 on an h.264 stream, not a chance:

Compared to the original:

Any time there's fast motion, particles, or the camera is panning, it looks like absolute garbage! So, IMO this is a valid use case and it really needs fixing. Emby's playback of half of my media library is unwatchable otherwise and I've been stuck casting files directly from my media server to things like VLC media player. I shouldn't have to do this.

 

I have to say I'm disappointed in this software's capabilities today, especially for something that, when I originally chose it, was open source and less restrictive in general. Maybe I need to reconsider Emby as my media server of choice if the dev team isn't even going to cover a dead simple, straightforward use case like this. It's not like I'm asking to move mountains, the team literally did work somewhere in the code to *block* customization they exposed in the first place. I understand if it's "default" behavior to limit the filesize for the typical user, but if I put in a custom CRF value that should override it.

Edited September 14, 2024 by drapacioli

[drapacioli 8]

For the record I was also looking into the REST api documentation to just fix it myself with a server side script or something, but it doesn't look like any sort of hook or listener event for client playback is included, and I don't know my way around an SDK so that's simply out of my league, but if anyone has the knowledge to either show me what to do or something, I'm all ears. Would be happy to learn, but I have only a very basic, self-taught understanding of simple things like python scripts and API endpoints, the SDK documentation in general just makes my head spin. IDK what I'm looking at or even what the "simple" examples are doing other than, well, existing I guess. I also tried peeking at the diagnostic options plugin, but it's encrypted or something I can't see what it's actually doing at all. Shame because that would probably have been the exact bit of code I'd need to modify for my own needs. 

 

I also tried just adding a few wildcard types like regex or simple wildcards in that text to replace field, sadly none of that works either.

Edited September 14, 2024 by drapacioli

[drapacioli 8]

I've been completely unable to get a practical working solution here on Emby. For anyone looking for help with the same issue, the only meaningful solution I've been able to find so far was switching to Jellyfin instead for anime playback. For some reason, Jellyfin's hardware transcoding is cleaner which is funny because it's literally a fork of Emby and it's got the same bandwidth limitation, but I think it's transcoding to h.265 which may be the difference. Quality still not quite the same as original, but much better when the source is h.264. Emby seems to have the same box in transcoding advanced options to enable h.265 encoding, but it seems it does not use this option even when I disable h.264 encoding explicitly (Note this is a test run on a different windows box due to jellyfin and plex being unable to coexist on the same windows server, so it's nvidia vs intel encoders, but I would expect the behavior is the same as both hardware GPUs support h.265 encoding):



I hope Emby can resolve this issue in the future, in the meantime I'm now running two media servers instead of one to satisfy this use case...

Edited September 19, 2024 by drapacioli

[semioniy 4]

So, it's been almost a year. I installed a cheap Intel ARC A310 into my UnRaid server and was able to make transcoding work, but it caps out at 2Mbps for HEVC and 4Mbps for H.264 despite the client demanding 60Mbps of quality. Look at these beautiful artifacts

TBH, I don't quite understand where the settings for HWA transcoding are. Here's the Transcoding page:

There are certainly settings for Software Encoding, but HWA encoding can only be switched On/Off.
If I go to 'Advanced' instead of 'Yes', I see

The options cog only near the Preferred Hardware Encoders - is it here that I set my desired encoding quality? If yeas - then also no dice, setting Encoding preset to Slow and CRF to 10 changes absolutely nothing, I still get the same bitrate.

 

So, where do I change the quality?

Edited June 26, 2025 by semioniy

[drapacioli 8]

2 minutes ago, semioniy said:

So, where do I change the quality?

That's the neat part, we don't...

 

I've ultimately resorted to effectively tripling my used storage space by re-encoding things in higher quality than necessary where I need to be able to swap between subs, and burning in where I can. It was $300 for the hard drive space to do it, sadly the emby team was not willing to budge on this design philosophy. Even the solution I thought I had before - Jellyfin - has other limitations that made it impractical. 

[RanmaCanada 494]

Why do people insist on using a web browser when Emby Theatre exists..

[semioniy 4]

Let's keep the replies on topic please.

[semioniy 4]

Here's my attempt to test if smth is wrong with my GPU - I installed Jellyfin and set subtitles to burn in always. Despite the same bitrate, the quality seems much higher, so it's something in the transcoding settings that emby sets inefficiently (I guess).

Emby HEVC (unwatchable):

Emby H.264 (gradients are much better, but still somewhat blocky):

Jellyfin H.264:

Also, when seeking for a specific spot in player, emby needs 6-8 seconds to start playing when skipping to an unbuffered spot, while jellyfin starts playback right away. I will provide all the help to the emby team if they want to resolve this issue, which has been present from the day I tried HWA on Emby (2 years ago)

P.S. I can also create and give you credentials to remotely access my local emby if needed

Edited June 26, 2025 by semioniy

[semioniy 4]

I wouldn't call this result conclusive at all though - I redeployed everything, and emby and jellyfin switched places, the HWA transcode of emby looking better than that of jellyfin. I have no idea how and why, it just happened.

The end result is not the point though - what would be nice is having any amount of control over that end result, instead of relying on that magical "backend will handle it somehow" thing

My offer stands though: I've spun up an extra instance of emby for testing, as well as a jellyfin for comparison, and can give access to it if you need an edge-case to test - Running on UnRaid in docker, using Intel ARC GPU for transcoding. Access to configs / logs can also be done over nextcloud, I just need to create users for all that and send credentials.

Edited June 26, 2025 by semioniy
added images

[semioniy 4]

Hello. Is there an intention to do something about HWA Encoding having no quality settings? (as well as needing 8 seconds to start playback when HWA is enabled)
It's a burning topic for me, so I couldn't just wait for a reply

[sh0rty 717]

2 hours ago, semioniy said:

Are you on Beta server?

 

Edit: Nevermind, found the thread.

Edited July 1, 2025 by sh0rty

[semioniy 4]

Morning. Any response from the devs?

[Luke 42077]

On 7/1/2025 at 6:25 AM, semioniy said:

Hello. Is there an intention to do something about HWA Encoding having no quality settings? (as well as needing 8 seconds to start playback when HWA is enabled)
It's a burning topic for me, so I couldn't just wait for a reply

 

Hi there, let's look at an example. Please attach the information requested in how to report a media playback issue. Thanks!

 

[semioniy 4]

Hi @Luke. I'm not reporting a media playback issue though — media plays back, the problem is that I can't control in what quality it transcodes, when using HWA transcoding.

[Luke 42077]

On 7/19/2025 at 5:44 PM, semioniy said:

Hi @Luke. I'm not reporting a media playback issue though — media plays back, the problem is that I can't control in what quality it transcodes, when using HWA transcoding.

@semioniyright I understand, but the same information that you would normally provide for a playback issue would also help us here. Can you please provide that. Thanks !