Hardware transcoding not working on any player (Emby Theatre, web browser, TV etc.)

[allanp81 0]

I'm using Emby via a docker on my unraid server and have passed through my Nvidia 2060 for hardware transcoding. I see this available within the transcoding settings and can enable hardware transcoding for all of the codecs but it never works. It seems to generate multiple logs and then eventually fails but I can't see an obvious reason as to why.

If I disable hardware decoding it seems everything then works fine but even with my Xeon CPU it uses a lot of power to achieve this so would prefer to make use of the GPU transcoding. It doesn't seem to matter what file I attempt to playback either.

Direct play works fine on whatever I try so it seems to only be a hardware transcoding problem.

embyserver.txt ffmpeg-transcode-1425ccd1-6139-4644-b8dd-0268956cc757_1.txt ffmpeg-transcode-de971d30-36ef-4e92-85d6-b65a8d8d4aa1_1.txt ffmpeg-remux-4613fac4-6520-45f8-aafc-3c9fe57b2735_1.txt ffmpeg-transcode-5f3b1eb6-ad09-4217-938f-a6d1a0d0a7fd_1.txt ffmpeg-remux-fc254bd5-7cbd-4a4e-85de-0b4564cfed99_1.txt

[Abobader 3290]

Hello allanp81,

** This is an auto reply **

Please wait for someone from staff support or our members to reply to you.

It's recommended to provide more info, as it explain in this thread:

Thank you.

Emby Team

[allanp81 0]

As a follow up to this, I've tried Jellyfin and HW transcoding on that works fine so this appears to be an Emby/FFMPEG problem.

[visproduction 231]

RTX 2060 does not support AVI 4:2:0 8 or 10 bit.  Otherwise, it should be able to do HEVC. 

 https://en.wikipedia.org/wiki/Nvidia_NVDEC

Do you have the latest drivers?

 https://www.nvidia.com/en-us/drivers/

 

[allanp81 0]

Yes, I have the latest drivers. I'm using via unraid in a docker.

[allanp81 0]

I don't know if there's any significance to the error "Failed to initialize VA /dev/dri/renderD128" that I see in the hardware detection log when I start the docker?

 

[yocker 135]

46 minutes ago, allanp81 said:

I don't know if there's any significance to the error "Failed to initialize VA /dev/dri/renderD128" that I see in the hardware detection log when I start the docker?

 

Have you put in --runtime=nvidia in the docker container?
/dev/dri/renderD128 is afaik only for Intel so if you put that in the docker container your Nvidia card won't work.

[allanp81 0]

Yes, I've got the runtime part and I have the nvidia visible devices etc. It seems that it might only be a problem with HEVC files, namely 4k films I've ripped from disc. Same files work fine transcoding in hardware on jellyfin.

[allanp81 0]

Here is my docker settings:

 

 

I've tried setting the GIDLIST as 18 and other things but makes no difference. HEVC transcoding works in Jellyfin etc. but not this. I'm also using the beta as it wouldn't work with the normal docker so I thought maybe it was fixed in the beta and then when I tried to go back to final it wouldn't because of a DB error.

[allanp81 0]

Any help would be appreciated, I paid a chunk of money for Emby premiere so would be really nice to get this working.

[Neminem 762]

This worked for me.

Try adding --gpus all to extra parameters

[allanp81 0]

Thanks for the suggestion but sadly that no made no difference.

[Neminem 762]

It was worth a try.

I see this in all your logs.

14:29:15.817   Stream #0:0 -> #1:1 (copy)
14:29:15.817 Press [q] to stop, [?] for help
14:29:15.819 subtitle_kickoff: call subtitle_resend_current 7090218 frame->format: 3
14:29:15.826 subtitle input filter: decoding size 384x288>> ThrottleBySegmentRequest: Latest request position unknown
>> ThrottleBySegmentRequest: Latest request position unknown
>> ThrottleBySegmentRequest: Latest request position unknown
>> ThrottleBySegmentRequest: Latest request position unknown
>> ThrottleBySegmentRequest: Latest request position unknown

Can you try playing without subs.

 

Edited March 9 by Neminem

[allanp81 0]

I have, seems to make no difference. My BluRay rips work fine, seems to only be a problem with HEVC rips. 

[visproduction 231]

Suspect that Blu-ray SUP (Subtitles Picture) may be in your HEVC rips.  If true and tagged as active, converting them to burned subs at the same time as video transcoding, could be part of the issue.  A sup may need to be resized as well and running both video encode plus the sup resize encode may just be causing the failure.  Try removing the subtitles from HEVC with mkvtools or other app and use converted subs in .srt or .vtt instead.  This may not be a good solution, depending on your number of media files that would need to be adjusted, but the test can tell you where the problem lies.

Hope that helps.

[allanp81 0]

I'm not sure, comparing a bluray remux versus a 4k uhd remux I'm not really seeing any difference other than the video codec is HEVC versus h.264.

I've tried taking my remux and removing all subs and it still fails.

[Luke 39294]

On 3/10/2025 at 6:26 AM, allanp81 said:

I'm not sure, comparing a bluray remux versus a 4k uhd remux I'm not really seeing any difference other than the video codec is HEVC versus h.264.

I've tried taking my remux and removing all subs and it still fails.

Hi, can we please see an example of that?

How to Report a Problem

Thanks !

[allanp81 0]

Here are the logs trying to play a file back with no subtitles included.

embyserverlogs.zip

[visproduction 231]

Similar issue?

From your 2nd ffmpeg-transcode...b36_1.txt

line 127

Quote

11:51:54.728 [hevc @ 0x11b4dc0] Skipping NAL unit 63

see: https://forum.jellyfin.org/t-solved-not-possible-to-play-media-when-nvidia-nvenc?page=2
 

Quote

You did not enable hardware accelerated DECODING. The list of check boxes at the top. Meaning the CPU is decoding the video.

Use this table to check the correct boxes.

https://developer.nvidia.com/video-encod...matrix-new
Jellyfin 10.10.6 (Docker)
Ubuntu 24.04.2 LTS w/HWE
Intel i3 12100
Intel Arc A380
OS drive - SK Hynix P41 1TB
Storage
    4x WD Red Pro 6TB CMR in RAIDZ1

Also see search: https://duckduckgo.com/?q=Skipping+NAL+unit+63+video+encoding+error&ia=web

===
No subs, but a lot of audio...

Also noticed that with no subs, the First and Second audio are DTS-HD stereo and DTS-HD Surround, then two audios with AC3 Stereo.  I believe all these will typically not play directly and need transcoding for many hardware and TV's.

===
embyserver.txt

Trying to playback to Chrome Windows.  I would assume video and audio, all need transcoding.  Not sure what is happening here.  It looks important.  Ha!

line 646-648

Quote

2025-03-14 11:52:02.767 Debug App: Request StreamTranscode 63dc5f cancelled during ValidateEncoderOutput
2025-03-14 11:52:02.782 Debug SqliteItemRepository: GetitemById Movie 26024 /data/content/user/P/Video/4K/Dune (1984) tt0087182/DUNE - 4K UHD.mkv
2025-03-14 11:52:02.783 Info SessionManager: Playback stopped reported by app Emby Web 4.8.11.0 on Chrome Windows playing Dune. Stopped at 0 ms. PlaySessionId: 448e47335b8a4bf2a69153ef8264d982
 

Hope that helps.

[allanp81 0]

All boxes for hardware transcoding within Emby are ticked:

 

All of the files I attempt with Emby work fine in Jellyfin (and used to work fine in Emby).

 

I fully expect some video/audio codecs to require transcoding. That's not the issue, the issue is that they never even start playing with hardware transcoding enabled. Everything works fine with software transcoding.