Nvidia HW transcoding fails for some files

[unmovable 11]

Transcoding of some files with NVIDIA HW decoder fails. Video and audio is stuttering / freezing and following errors appear in ffmpeg logs:

03:51:03.429 [hevc @ 0xccbd00] Could not find ref with POC 26
03:51:03.429 [hevc @ 0xccbd00] No decoder surfaces left
03:51:03.429 [hevc @ 0xccbd00] decoder->cvdl->cuvidDecodePicture(decoder->decoder, &ctx->pic_params) failed -> CUDA_ERROR_INVALID_VALUE: invalid argument
03:51:03.429 [hevc @ 0xccbd00] hardware accelerator failed to decode picture
03:51:03.429 Error while decoding stream #0:0: Generic error in an external library

Transcoding settings:

 

Emby server: Version 4.8.0.21 beta (docker linuxserver)

 NVIDIA Driver version: v525.60.13

ffmpeg-transcode-2c22c8af-0b7d-4605-bd9a-b9b991963160_1.txt

Edited January 26, 2023 by unmovable

[Luke 37068]

Hi, can you try our official docker container for comparison purposes? How did you configure the container?

[Luke 37068]

For example with our container we have guidance on configuring gpu's:

https://hub.docker.com/r/emby/embyserver

[unmovable 11]

@Luke Tried with the official container, got same results. Container is configured as per the instructions on the link and HW transcoding is working properly on other files (both, for x265 and x264), but fails for this title (all episodes). Perhaps there's something in this encode that makes emby to generate improper ffmpeg parameters?

Also, same files work properly on direct play capable devices, it's just the ones where transcoding is necessary where it fails.

[Luke 37068]

Can you please provide the emby server and ffmpeg logs from our container, as well as what your docker run command looks like? thanks.

[unmovable 11]

25 minutes ago, Luke said:

Can you please provide the emby server and ffmpeg logs from our container, as well as what your docker run command looks like? thanks.

Here are the embyserver, ffmpeg and hwdetect logs:

https://gist.github.com/grapexy/d48b200ae64af82a7e782000c7c27ae7

Docker run:

docker run
  -d
  --name='emby'
  --net='br0.40'
  --ip='192.168.40.30'
  --cpuset-cpus='1,2,3,4,5,6,7,9,10,11,12,13,14,15'
  -e 'UID'='99'
  -e 'GID'='100'
  -e 'UMASK'='022'
  -e 'GIDLIST'='100'
  -v '/mnt/mergerfs/media/tv/':'/data/tvshows':'ro'
  -v '/mnt/mergerfs/media/movies/':'/data/movies':'ro'
  -v '/mnt/user/appdata/emby':'/config':'rw'
  -v '/mnt/user/media/data/movies/movieshd/':'/data/hd-movies':'ro'
  -v '/mnt/user/media/data/tvhd/':'/data/hd-tv':'ro'
  -v '/tmp/embytranscode':'/transcode':'rw'
  --device='/dev/dri'
  --runtime nvidia
  --gpus all 'emby/embyserver:beta'

 

[Luke 37068]

What OS are you on? As a test, can you install our native package for your OS and see how that compares?

[unmovable 11]

OS is Unraid 6.11.5. I believe getting emby running natively on Unraid will be a close to impossible task, since there are no slackware packages.

[unmovable 11]

@Luke After some investigation, I think this is affecting all 8-bit HEVC files

Scratch that. Some 8-bit ones do transcode just fine. Also, same file transcodes fine on CPU. This seems to also be affecting some Plex users recently - https://forums.plex.tv/t/hardware-transcoding-issue/826410

Edited January 26, 2023 by unmovable

[unmovable 11]

Unchecking "NVDEC Quadro P400 - H.265 (HEVC)" resolved the issue and GPU is still used according to both emby dashboard and nvidia-smi. This got more confusing, but at least it works for now.

 

[unmovable 11]

Scratch the last post. Unchecking it disables HW transcoding, dashboard and nvidia-smi are lying, as CPU usage goes through the roof.

1080p HEVC transcoding is essentially broken on beta.

 

@Luke is this being addressed?

Edited January 30, 2023 by unmovable

[Luke 37068]

Hi, can you provide a sample video for testing? Thanks.

[unmovable 11]

37 minutes ago, Luke said:

Hi, can you provide a sample video for testing? Thanks.

PM sent

[cbusch 1]

I hate to me too on this, but me too. I'm getting many files failing with "No decoder surfaces left" when hardware transcoding remotely.

ffmpeg log attached.

ffmpeg-transcode-a139aee1-3be7-46a0-b3e3-347eb8b31b1d_1.txt

[cbusch 1]

My docker compose and embyserver log.

version: "2.3"
services:
  emby:
    image: emby/embyserver:beta
    container_name: embyserver
    runtime: nvidia # Expose NVIDIA GPUs
    network_mode: host # Enable DLNA and Wake-on-Lan
    environment:
      - UID=850 # The UID to run emby as (default: 2)
      - GID=1002 # The GID to run emby as (default 2)
      - GIDLIST=44,109 # A comma-separated list of additional GIDs to run emby as (default: 2)
    volumes:
      - /opt/emby/programdata:/config # Configuration directory
      - /opt/emby/intros:/mnt/intros # custom Intros
      - /opt/emby/backups:/backups # Backups
      - /media/vault/tv:/mnt/tv # Media directory
      - /media/vault/movies:/mnt/movies # Media directory
    ports:
      - 8096:8096 # HTTP port
      - 8920:8920 # HTTPS port
    devices:
      - /dev/dri:/dev/dri # VAAPI/NVDEC/NVENC render nodes
    restart: unless-stopped

 

embyserver.txt

[cbusch 1]

@Luke were you able to figure out anything to help @unmovable with the same issue?

[Happy2Play 8282]

Several topics in Testing area but if you use CUVID instead of NVDEC do you still have an issue?

Issue with hardware transcoding on 4.8.0.21 - Page 2 - Emby Server - Emby Community

 

[cbusch 1]

Damn, well I feel dumb. That's my bad not checking the Testing section. Yep, switching to CUVID seems to have fixed the problem for the few problematic files I tested.

Thanks!