Rotated videos force full software transcode on NVIDIA. HW decode is dropped, playback fails.

[danergo 48]

Environment

• Emby Server 4.9.5.0, Linux (Ubuntu, kernel 7.0.0-38-generic), x64
• Bundled ffmpeg: 5.1-emby_2023_06_25_p4
• GPU: NVIDIA Quadro P400 (Pascal, CC 6.1), detected correctly. NVDEC/CUVID HEVC decoders and NVENC H.264 encoder are all listed and pass checks.
• Hardware acceleration mode: Advanced
• Client: Emby Web 4.9.5.0, Chrome on Windows, remote connection

Source file

• DJI Osmo Action 4 recording, MP4, 3840x2160, HEVC Main 10 (yuv420p10le), 50 fps, ~110 Mbit/s, Level 150, SDR/BT.709
• The camera was mounted upside down, so the file carries rotation metadata:

  Side data:
    displaymatrix: rotation of -180.00 degrees

Problem

Playback fails with “No compatible streams are currently available.” Non-rotated 4K HEVC files from the same camera transcode fine on the same server with full NVDEC → NVENC.

The transcode does start, but runs at about 6 fps (`speed=0.11x`) for a 50 fps source. The client gives up and stops it (`[q] command received`):

elapsed=00:00:09.54 frame=58 fps=6.1 ... speed=0.111x
[q] command received. Exiting.

Analysis

The processing plan shows rotation is not supported in hardware:

Name                                CanDoInHW  WillDoInHW
NVDEC Quadro P400 - H.265 (HEVC) >> True       True
VideoInput                       >> True       False
Rotation                         >> False      False
Scaling                          >> True       True
ColorConversion                  >> True       True
VideoOutput                      >> True       True
NVENC Quadro P400 - H.264 (AVC)  >> True       True

The resulting filter chain rotates in software before uploading to CUDA:

HEVC >> rotate >> hwupload_cuda >> scale_cuda >> setsar

The plan says NVDEC will be used, but the actual ffmpeg command line uses the software HEVC decoder limited to a single thread. There is no `-hwaccel cuda` and no `hevc_cuvid`:

-c:v:0 hevc -threads:v:0 1 -noautorotate ...
-filter_complex "[0:0]rotate@f1=angle=180*(PI/180),hwupload_cuda@f2,scale_cuda@f3=w=1920:h=1080:format=yuv420p,setsar@f4=sar=sar[f4_out0]"

So a single CPU thread has to decode 4K 10-bit HEVC at 110 Mbit/s, and then a generic software `rotate` filter runs on every 4K 10-bit frame. NVENC encoding itself works fine. The bottleneck is entirely the forced software decode plus software rotation.

Why this seems unnecessary

1. The hardware can do it. A 180° rotation is just a horizontal plus vertical flip. The decoded frame is already in GPU memory after NVDEC, so a trivial CUDA operation (or NPP, e.g. `transpose_npp`) could handle it without leaving the GPU.
2. Even if rotation must stay in software, the fallback could be much better. The pipeline could still decode on NVDEC, `hwdownload`, flip on the CPU, and `hwupload_cuda` again. Failing that, it could at least use a multi-threaded software decoder instead of `-threads 1`.
3. The software `rotate` filter is a poor choice for 180°. It is a general arbitrary-angle rotation with interpolation. `hflip,vflip` (or `transpose`-based equivalents) would be considerably cheaper for 90/180/270° cases.

Requests

• Add hardware rotation support for NVIDIA (CUDA/NPP-based flip/transpose), at least for 90/180/270°, so rotated videos can stay on a full HW pipeline.
• Until then, when rotation forces a software step, keep hardware decoding and only download/upload around the rotation step, or don’t limit the software decoder to one thread.
• Use `hflip`/`vflip`/`transpose` instead of the generic `rotate` filter for right-angle rotations.

This affects a very common use case: action cameras (DJI, GoPro) and phones often record with rotation metadata, especially when mounted upside down.

Workaround for now: re-encode the affected files outside Emby, so the rotation is baked into the pixels and the metadata tag is removed.

Full transcode log available on request.

[Luke 43402]

 

Hi there, let's look at an example. Please attach the information requested in how to report a media playback issue. Thanks!

 

[danergo 48]

@Luke, this is the error I'm seeing:

Please find the logs attached.

ffmpeg-transcode-3f43cb06-7e9b-452f-b29f-e83969546f98_1.txt embyserver (12).txt

[danergo 48]

Hi @Luke, do you need anything from my side? Please let me know.