AMD 5600 XT unable to use hardware acceleration/tone mapping

[encode42 0]

I'm using Emby 4.9.1.90 on NixOS 25.11 (not very supported, I know!) using a flake I found online, forked and updated. You can view how Emby is packaged in this flake, and the module creating the systemd service here. This flake could very well be broken and/or the cause of this issue! However, Emby works just fine otherwise, and after a bunch of debugging, I couldn't pinpoint the issue directly onto the flake. I can edit the package or the module as needed for testing. Below is the resulting service:

Spoiler

[Unit]
After=network-online.target
Description=Emby Media Server
Wants=network-online.target

[Service]
Environment="LIBVA_DRIVER_NAME=radeonsi"
Environment="LOCALE_ARCHIVE=/nix/store/9vmxdn26pggj3glazmb68vhcf4smflvz-glibc-locales-2.40-66/lib/locale/locale-archive"
Environment="PATH=/nix/store/imad8dvhp77h0pjbckp6wvmnyhp8dpgg-coreutils-9.8/bin:/nix/store/av4xw9f56xlx5pgv862wabfif6m1yc0a-findutils-4.10.0/bin:/nix/store/x3zjxxz8m4ki88axp0gn8q8m6bldybba-gnugrep-3.12/bin:/nix/store/drc7kang929jaza6cy9zdx10s4gw1z5p-gnused-4.9/bin:/nix/store/zf8qy81dsw1vqwgh9p9n2h40s1k0g2l1-systemd-258.2/bin:/nix/store/imad8dvhp77h0pjbckp6wvmnyhp8dpgg-coreutils-9.8/sbin:/nix/store/av4xw9f56xlx5pgv862wabfif6m1yc0a-findutils-4.10.0/sbin:/nix/store/x3zjxxz8m4ki88axp0gn8q8m6bldybba-gnugrep-3.12/sbin:/nix/store/drc7kang929jaza6cy9zdx10s4gw1z5p-gnused-4.9/sbin:/nix/store/zf8qy81dsw1vqwgh9p9n2h40s1k0g2l1-systemd-258.2/sbin"
Environment="TZDIR=/nix/store/xaa75rd44q62nc9mrbvym9d1m6gy0fj8-tzdata-2025b/share/zoneinfo"
Environment="VDPAU_DRIVER=radeonsi"
DeviceAllow=/dev/dri/card0
DeviceAllow=/dev/dri/renderD128
ExecStart=/nix/store/k6p68j9ppkj5lmbrqmbv2zn57fwl3d2x-emby-server-4.9.1.90/bin/emby -programdata '/mnt/apps/emby'
Group=emby
LockPersonality=true
NoNewPrivileges=true
PrivateDevices=false
PrivateTmp=true
PrivateUsers=true
ProtectControlGroups=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
RemoveIPC=true
Restart=on-failure
RestrictAddressFamilies=AF_UNIX
RestrictAddressFamilies=AF_INET
RestrictAddressFamilies=AF_INET6
RestrictAddressFamilies=AF_NETLINK
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
StateDirectory=emby
SuccessExitStatus=0
SuccessExitStatus=143
SystemCallArchitectures=native
SystemCallErrorNumber=EPERM
TimeoutSec=15
Type=simple
UMask=0077
User=emby
WorkingDirectory=/mnt/apps/emby

[Install]
WantedBy=multi-user.target

When playing back HDR video, colors appear very "washed out" - as if everything is more gray. The scrubbing preview images, however, appear to have the correct colors. The below screenshot was taken using Firefox 145.0.2 on NixOS 25.11 once again (a different machine). The display does not support HDR.

 

The machine bears an AMD Ryzen 7 5700X paired with an AMD RX 5600 XT. VAAPI works on this machine, and other services such as Immich have been able to utilize hardware acceleration:

Trying display: wayland
Trying display: x11
error: can't connect to X server!
Trying display: drm
libva info: VA-API version 1.22.0
libva info: Trying to open /run/opengl-driver/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Mesa Gallium driver 25.2.6 for AMD Radeon RX 5600 XT (radeonsi, navi10, LLVM 21.1.2, DRM 3.61, 6.12.56-hardened1)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

This shows in Emby as follows: 

And tone mapping is set to Extra-T, the only option:

Despite this, I don't believe hardware acceleration is ever actually in use. Playback always pins my CPU to full usage, and no tone mapping algorithm is applied. No change in the tone mapping algorithm appears to have any effect. This occurred on Emby 4.8.11.0 as well, and the HEVC encoding option doesn't have an impact. I've attached the hardware detection result, as well as the ffmpeg logs when playing back a movie with hardware acceleration and tone mapping supposedly enabled. Of course, I can provide any details as requested.

 

I noticed that post #144043 had a similar issue to me, however it appears they're using QSV as well, so I decided to create a new post.

hardware_detection-63900295045.txt ffmpeg-remux-a53a4d15-2e09-4a76-a1a6-8fc1c644cd69_1.txt embyserver.txt

Edited December 2, 2025 by encode42
Weird formatting issues + systemd service

[encode42 0]

Looking at the "Now Playing" dashboard section while I'm reproducing this issue, it appears that tone mapping might not even be occurring? I won't lie, I'm not very knowledgeable in media playback - so I can't make any well-educated guesses.

[Lessaj 467]

It's direct playing the video, there is no transcoding involved for the video so there's no tone mapping happening, only the audio is transcoded to a different format. This means your browser isn't tone mapping like it should, how does Chrome/Chromium compare? Looks okay on my end (Chromium Linux).

 

[encode42 0]

I'm not sure why Firefox isn't calling for transcoding, however it appears that Chromium is: (definitely less washed out than before) 

 

Even still, though, I don't believe Emby is using hardware acceleration. Interestingly, software tone mapping is currently disabled. The hardware tone mapping setting doesn't appear to change the video, and Emby is hammering my CPU while the GPU just doesn't care.

 

Edited December 5, 2025 by encode42
why was another screenshot added?

[Lessaj 467]

That's unfortunately not an apples to apples test unless it is direct playing the video, I believe you'll have to add some flags in order to get GPU acceleration to work in Chrome so that HEVC can be direct played. It's definitely not using hardware acceleration since there's no icon, and I don't think AMD GPUs can do tone mapping in hardware, I could be wrong. It looks like it is doing software tone mapping otherwise it would look washed out still, but would need to see the ffmpeg transcode logs to understand more since you said that's disabled.

I'm using these flags and HEVC decoding works for me - I don't remember if I had to also toggle "Use graphics acceleration when available" or if it was already enabled. Go to chrome://gpu and you're looking for "Decode hevc main 10" under Video Acceleration Information, otherwise it probably won't work. I've had quite the adventure with getting hardware acceleration for HEVC to work, at one point it reported it was supported but I just got a black screen with audio so YMMY on the exact flags you need to use for your hardware.

--use-gl=angle
--use-vulkan=native
--use-angle=vulkan
--enable-features=Vulkan,VulkanFromANGLE,DefaultANGLEVulkan,VaapiIgnoreDriverChecks,AcceleratedVideoDecoder,UseMultiPlaneFormatForHardwareVideo,AcceleratedVideoEncoder
--ignore-gpu-blocklist
--enable-oop-rasterization
--force-dark-mode
--enable-zero-copy
--enable-gpu-rasterization
--enable-native-gpu-memory-buffers
--enable-gpu-memory-buffer-video-frames
--disable-gpu-driver-bug-workarounds

 

[encode42 0]

Here's the FFmpeg transcode logs from the above test: ffmpeg-transcode-9e2bbfc7-bccb-4bef-8ce1-383c50100b76_1.txt

I should clarify that I don't really care whether the client is using hardware acceleration or not (I'll deal with that later), rather the important side is the server. I've got quite a lot running on the server machine, so I can't afford Emby transcoding through the CPU. Supposedly, judging by Emby's transcoding settings, it should be able to transcode + tone map using the server's AMD card through VAAPI and OpenCL, which it's currently configured to use.

Despite this, Emby doesn't seem to be utilizing the GPU at all. I'm not sure whether it's missing some system libraries (due to NixOS's sandboxing), running into an unusual permissions or hardening issue (again due to NixOS's sandboxing), or if my card just isn't supported.

Edited December 5, 2025 by encode42
grammar

[Lessaj 467]

It was more just to see if the behaviour is specific to firefox or not, which it might be, but yes certainly want to have hardware transcoding working and performing the tone mapping. The problem is Firefox isn't tone mapping even though it's reporting that it supports the codec. What about if you use playback correction? Does that end up transcoding using the GPU?

In the case attached I believe it is performing software tone mapping, even though it does say it's disabled, since it's in the processing plan. It can do it fast enough which is good actually, but at the cost of max CPU usage so I agree that's not ideal. I'd recommend enabling throttling if it's not enabled, this can minimize the 100% CPU in the cases where it's performing audio transcoding because it will process the whole thing until it finishes without that on - I think that's all it was doing in your initial example since it's direct playing the video, I just couldn't tell if throttling was enabled since it was stopped early.

Quote

Info    Tone Mapping would be desired, but software tone mapping is disabled

>>>>>>  Processing Plan
        Name                                        CanDoInHW  WillDoInHW  Reason                                                 
        Automatic software decoder               >> False      False       Software Codec                                          
        VideoInput                               >> False      False       Not a hardware decoder                                  
        ToneMapping (when possible)              >> False      False                                                               
        VideoOutput                              >> False      False       Not a hardware encoder                                  
        x264                                     >> False      False       Software Codec   

[encode42 0]

Unfortunately, it doesn't look like playback correction has made a difference. Interestingly, even with throttling enabled, the server's CPU usage is still maxed by Emby when transcoding. It looks like direct video play (back to Firefox, throttling disabled) only uses one entire core rather than all of them (likely due to the audio transcoding), which is just fine.

When playing back media that doesn't have HDR (using Animusic HD which still requires transcoding for de-interlacing), Emby still uses a noticeable amount of CPU, but it causes significantly less load. About one entire core, similar the direct stream and audio, this time even while transcoding video as well. Granted, this test media is much older. Also very interestingly, the GPU is experiencing some usage this time? That would likely explain it - CPU for audio, GPU for video.

So, Emby is using hardware acceleration for transcoding, sometimes? Why isn't hardware acceleration working for tone mapping? Maybe the Extra-T OpenCL implementation is busted for my card? I've attached the transcoding logs for my latest test with SDR media.

ffmpeg-transcode-5660b302-ef6b-450f-83dd-be19f4bdbd68_1.txt

Edited December 5, 2025 by encode42
expand upon my thoughts

[Lessaj 467]

10 hours ago, encode42 said:

Unfortunately, it doesn't look like playback correction has made a difference.

Can you provide the ffmpeg log from that? Note that more than one log may be generated, if it tries HW accel and fails it might create a new log for SW transcode.

In the SDR example you provided it does look like your hardware acceleration is working via vaapi, and yes you should see some CPU usage since it has to transcode the audio as well.

Quote

02:01:35.676   Stream #0:0 (h264) -> deinterlace_vaapi:default (graph 0)
02:01:35.676   deinterlace_vaapi:default (graph 0) -> Stream #0:0 (h264_vaapi)
02:01:35.676   Stream #0:3 -> #0:1 (pcm_s24le (native) -> aac (native))

 

[Neminem 1518]

I'm wondering is the sheer number of audio tracks and subs embedded in /media/Movies/Blade Runner 2049 (2017)/Blade Runner.mkv

ffmpeg-transcode-9e2bbfc7-bccb-4bef-8ce1-383c50100b76_1.txt

Is the issue.

The total track number is 45.

And just to add to that Subs are PGS subs.

Not sure about this.

[Lessaj 467]

24 minutes ago, Neminem said:

I'm wondering is the sheer number of audio tracks and subs embedded in /media/Movies/Blade Runner 2049 (2017)/Blade Runner.mkv

ffmpeg-transcode-9e2bbfc7-bccb-4bef-8ce1-383c50100b76_1.txt

Is the issue.

The total track number is 45.

And just to add to that Subs are PGS subs.

Not sure about this.

Likely not relevant.

[encode42 0]

Back on Firefox, I started playback and let it run for ~a minute as usual. Then, I enabled playback correction and let that play. These are the FFmpeg logs generated during that period in order: ffmpeg-remux-2f5e3bfb-fa1e-4690-bae7-feac89ad2f78_1.txtffmpeg-transcode-127d1cbf-575c-44b4-9b3b-19cbb1b3a3d0_1.txt

I also noticed this hardware acceleration icon during playback correction. The GPU was utilized a bit during that period, however the CPU usage was still maxed out. It looks like this icon only pertains to hardware accelerated decoding, which is nice, but not quite what I'm looking for.

[Lessaj 467]

Okay I see an error in the transcode log now, this is why it's not using hardware acceleration for the encode. And this cannot process fast enough on SW encoder, only around 0.6x.

Quote

>>>>>>  FindVideoEncoder - MediaType: hevc, UseHardwareCodecs: True, HWA-Mode: Advanced
Info    Checking: 'VAAPI Navi 10 Radeon RX 5600 - H.265 (HEVC)'
NoMatch Bitrate (60 Mbit/s) exceeds maximum supported rate (59 Mbit/s)
NoMatch Encoder does not match
Info    Checking: 'x265'
Info    Check successful - selecting 'x265'

Can you try disabling the experimental HEVC transcoding and see if that makes a difference?

Quote

>>>>>>  Hardware Decoders for hevc
        [X] VAAPI Navi 10 Radeon RX 5600 - H.265 (HEVC)

>>>>>>  Hardware Encoders for hevc
        [X] VAAPI Navi 10 Radeon RX 5600 - H.265 (HEVC)

Edited December 6, 2025 by Lessaj

[encode42 0]

Same situation as before, now with HEVC disabled: ffmpeg-remux-a9d4b81e-e24f-4cfb-8dc0-523f6f827632_1.txtffmpeg-transcode-26fd0cdf-23df-43cb-bef7-357a0e6d8274_1.txt

Unfortunately, tone mapping still isn't working, and the CPU is being pinned. Also interestingly, no GPU utilization this time.

Edited December 7, 2025 by encode42
words

[Lessaj 467]

I can see it's failing when attempting with hardware, which I think is what I mentioned earlier that AMD GPUs cannot do tone mapping, since we saw the hardware acceleration working with SDR content. I may be incorrect on that, if that situation has changed. It falls back to software which it can do in real time over 3x so with throttling enabled it would run for a while to build the ~2 minute buffer and then it would be on and off usage - it says tone mapping is disabled in software.

21:42:08.048 [AVHWDeviceContext @ 0x20cd7d80] Failed to get number of OpenCL platforms: -1001.
21:42:08.048 [hwmap@f1 @ 0x20f94b00] Failed to created derived device context: -19.
21:42:08.048 [hwmap@f1 @ 0x20f94b00] Failed to configure output pad on hwmap@f1
21:42:08.048 Error reinitializing filters!
21:42:08.048 Failed to inject frame into filter network: No such device
21:42:08.048 Error while processing the decoded data for stream #0:0
21:42:08.048 [aac @ 0x20f44d00] Qavg: nan
21:42:08.052 Conversion failed!

 

Edited December 7, 2025 by Lessaj

[Luke 42077]

Hi, we’ll have a new ffmpeg build on the server beta channel in the near future, so please stay tuned for that. Thanks.

[bunty_sam 0]

On 1/4/2026 at 11:58 PM, Luke said:

Hi, we’ll have a new ffmpeg build on the server beta channel in the near future, so please stay tuned for that. Thanks.

Hi I had a similar issue in the recent past. Will the beta version resolve this issue? If yes then I would like to test again by getting emby premiere.