Jump to content

HW Scaling not working


Go to solution Solved by studzik5521,

Recommended Posts

studzik5521
Posted

Hey, hardware scaling isn't working for me on my Nvidia card (GTX 1050, and it didn't work on the 750 either).
I had to enable the 'Disable Scaling' option in the 'Diagnostic Options' plugin to get hardware support for the rest to work.
image.png.b9d590d5e721b1a7ef9630575b81269a.png

Does anyone have an idea what could be causing this?

 

ffmpeg-transcode-69229d6c-1895-4b12-8961-df2fcbfbb879_1.txt

Posted

Hi there, please attach the corresponding emby server and hardware detection log files. Thanks.

visproduction
Posted

ffmpeg-transcode-fc...

3 x subrip conversions to .vtt at the same time

Also line 110+

13:08:55.493   Stream #0:4(pol): Subtitle: subrip (forced)
13:08:55.493     Metadata:
13:08:55.493       title           : Polish [Forced]
13:08:55.493       BPS             : 0
13:08:55.493       DURATION        : 00:49:15.452000000
13:08:55.493       NUMBER_OF_FRAMES: 5
13:08:55.493       NUMBER_OF_BYTES : 101

So only 5 instances of non-Polish subtitles and this forced sub has to be converted to .vtt.  So it will be scanning for only 5 lines in 49 minutes.  Does this cause an issue with subrip conversion.  This forced sub process looks like it will be looking for 10 minutes screen times with no subs at all inside the forced file.  How does the encoder react to this and do the other two suprip conversions at the same time?

Guessing this can clog the encoder pretty much.  Perhaps upgrade to how multiple subrip subs are handled? Do one only and delay or cancel other subrip conversions?  I would guess if you turn off these subrip conversions, or remove them and remux the media,  the encoding will work.  A lot of encoding software is set to ignore subtitles because such multiple conversions often cause encoding to fail.  In this case, two audios in ec3 that probably need conversion and 3 subrips, all at the same time.

studzik5521
Posted

This works when I turn off scaling, even with forced subtitles (ss below)
image.thumb.png.bb325f79c863569fb43a73298f8a4377.png
 

Also works in direct stream.image.thumb.png.37c636f1b8ce26331ea55026df045aac.png


Does not work when scaling from e.g. 1080p to 720p (with scaling enabled)

When I try to run a movie, the GPU usage does not increase and the CPU usage is 100% (Subtitles disabled) image.png.cf87e3c91d210bc45ae7744960b8dd70.png
image.png.6d935e534d8c3c4cecd8a67a35b6ff76.png


What could be the cause? Or what to do to diagnose it better?

studzik5521
Posted

This occurs in every movie/episode where image scaling occurs.

Can anyone help or advise?

visproduction
Posted

Also noticed in Embyserver(111).txt that Subtitles are being scaled to 640 x 360 because they are graphics based Subrip.  Since you have three of these subtitles and the main video that also needs scaling at the same time, I would guess you are running into a very intensive workload.  Pinning the CPU at 100% while doing all four scaling seems like it could be expected depending on your server setup.  

Subtitles disabled.  OK, what is happening exactly with the software?  Are these subrip graphic subtitles turned off?  Is there a default subtitle metatag in the media file, that forces one subtitle to be on?  Is this subtitle completely ignored when you tell the transcoding to not process?  Does ffmpeg still look at the subs?  If you really want to test to see if subtitles have an issue, turning them off is not really a test.  You need to use a test video with no subtitles and remove the subtitles completely, from your media and try that.

If you like metaphores, you may have just turned off the back lights for a trailer you are towing behind your truck, but the truck still doesn't go fast and there are noises coming from behind.  Right, try disconnecting the trailer, or find a smaller load that fits in the back of the truck.  

  • Haha 1
studzik5521
Posted

ok I'll try to "disconnect the trailer" or "lighten the truck" 😅

Now I understand that when I "turn off subtitles", I'm actually hiding them, not turning them off.
But Iit's strange that 3xsubtitles + movie scaling works on CPU but not on GPU, but I'll check it out.

I'll look for movies without subtitles and then I will upload the logs.

studzik5521
Posted (edited)

I don't know if it changes anything, but I use Emby Docker in an LXC container on Proxmox.

LXC:

root@docker-portainer:~# ls -la /dev/nvi*
crw-rw-rw- 1 root root 195,   0 Jan 29 01:29 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Jan 29 01:29 /dev/nvidiactl
crw-rw-rw- 1 root root 195, 254 Jan 29 01:29 /dev/nvidia-modeset
crw-rw-rw- 1 root root 234,   0 Jan 29 01:31 /dev/nvidia-uvm
---------- 1 root root        0 Jan 29 01:29 /dev/nvidia-uvm-tools

/dev/nvidia-caps:
total 0
drwxr-xr-x  2 root root     80 Jan 29 01:31 .
drwxr-xr-x 11 root root    760 Jan 29 01:31 ..
cr--------  1 root root 237, 1 Jan 29 01:31 nvidia-cap1
cr--r--r--  1 root root 237, 2 Jan 29 01:31 nvidia-cap2
root@docker-portainer:~# nvidia-smi
Wed Feb  5 15:49:19 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.142                Driver Version: 550.142        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1050        On  |   00000000:02:00.0 Off |                  N/A |
| 31%   31C    P8             N/A /   65W |       2MiB /   2048MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

 

Proxmox:

root@eddy:~# ls -la /dev/nvi*
crw-rw-rw- 1 root root 195,   0 Feb  5 15:51 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Feb  5 15:51 /dev/nvidiactl
crw-rw-rw- 1 root root 234,   0 Feb  5 15:51 /dev/nvidia-uvm
crw-rw-rw- 1 root root 234,   1 Feb  5 15:51 /dev/nvidia-uvm-tools

/dev/nvidia-caps:
total 0
drwxr-xr-x  2 root root     80 Feb  5 15:51 .
drwxr-xr-x 22 root root   4580 Feb  5 15:51 ..
cr--------  1 root root 237, 1 Feb  5 15:51 nvidia-cap1
cr--r--r--  1 root root 237, 2 Feb  5 15:51 nvidia-cap2
root@eddy:~# nvidia-smi
Wed Feb  5 15:52:55 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.142                Driver Version: 550.142        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1050        Off |   00000000:02:00.0 Off |                  N/A |
| 31%   31C    P8             N/A /   65W |       2MiB /   2048MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+



LXC Config:

root@eddy:~# cat /var/lib/lxc/102/config
lxc.cgroup.relative = 0
lxc.cgroup.dir.monitor = lxc.monitor/102
lxc.cgroup.dir.container = lxc/102
lxc.cgroup.dir.container.inner = ns
lxc.arch = amd64
lxc.include = /usr/share/lxc/config/debian.common.conf
lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1
lxc.monitor.unshare = 1
lxc.tty.max = 2
lxc.environment = TERM=linux
lxc.uts.name = docker-portainer
lxc.cgroup2.memory.max = 25769803776
lxc.cgroup2.memory.high = 25635586048
lxc.cgroup2.memory.swap.max = 0
lxc.rootfs.path = /var/lib/lxc/102/rootfs
lxc.net.0.type = veth
lxc.net.0.veth.pair = veth102i0
lxc.net.0.hwaddr = BC:24:11:D3:36:62
lxc.net.0.name = eth0
lxc.net.0.mtu = 1500
lxc.net.0.script.up = /usr/share/lxc/lxcnetaddbr
lxc.cgroup2.devices.allow = a
lxc.cap.drop =
lxc.cgroup2.devices.allow = c 188:* rwm
lxc.cgroup2.devices.allow = c 189:* rwm
lxc.mount.entry = /dev/serial/by-id  dev/serial/by-id  none bind,optional,create=dir
lxc.mount.entry = /dev/ttyUSB0       dev/ttyUSB0       none bind,optional,create=file
lxc.mount.entry = /dev/ttyUSB1       dev/ttyUSB1       none bind,optional,create=file
lxc.mount.entry = /dev/ttyACM0       dev/ttyACM0       none bind,optional,create=file
lxc.mount.entry = /dev/ttyACM1       dev/ttyACM1       none bind,optional,create=file
lxc.cgroup2.devices.allow = c 195:* rwm
lxc.cgroup2.devices.allow = c 234:* rwm
lxc.cgroup2.devices.allow = c 237:* rwm
lxc.cgroup2.devices.allow = c 255:* rwm
lxc.mount.entry = /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry = /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.cgroup.devices.allow = c 10:200 rwm
lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file
lxc.cgroup2.cpuset.cpus =

 

Edited by studzik5521
studzik5521
visproduction
Posted

Wow, that is a lot of info to go through.  Not sure if I have enough background to really comment on such transcoding.  I did notice that you run scale then Tone Map, then encode.  I would guess that running Tone Map before scaling might work better.  Any tone map process needs to recognize exact colors per pixel to make adjustments.  If you already scaled the media, then per pixel colors will no longer be the same.

studzik5521
Posted

These are the default settings, I didn't change anything, the application does it that way.

Tone map works. Nothing that uses "scale_cuda" works

studzik5521
Posted
On 2/6/2025 at 1:10 AM, studzik5521 said:

scale_cuda also does not work directly on proxmox (bare metal, debian) with a different driver version (previously 550.142 now 565.77)

There's a crash when ffmpeg tries to access CUDA.

How did you install the drivers?

studzik5521
Posted

Using the package from nvidia's website "NVIDIA-Linux-x86_64-565.77.run" or earlier "NVIDIA-Linux-x86_64-550.142.run"

sudo ./NVIDIA-Linux-x86_64-565.77.run

Posted

Hmm, that sounds all good.

Can you please install the Diagnostics Plugin and set ffmpeg logging to Debug.

Then reproduce once again (a single ffmpeg log is sufficient.

Thanks

Posted
40 minutes ago, studzik5521 said:

Hi, can you please perform the same test using the native package? Thanks.

Posted
1 hour ago, studzik5521 said:

Thanks a lot for the logs.

Unfortunately this doesn't give any further hint, other than that it confirms that it happens when initializing the CUDA filter.
It's not like anything I've seen before. It could be anything from hardware to OS, to drivers, to Proxmox (host).

Here are some things you can try to hopefully get a hint in the right direction. I would focus on CUDA functionality:

  • You could try a rather older version of drivers - like from 2022 or 2023
  • You can download the Nvidia CUDA toolkit.
    • There are a number of demos you can compile and run
    • And there's a tool included named "deviceQuery" which you can try to run
  • You can watch Linux logs (like dmesg - IIRC)
  • Trying a regular ffmpeg (with Nvidia HWA) might also be worth a try, but our HWA support for Nvidia on Linux is very stable and issue reports are rare
  • Finally you can try to contact Nvidia support, but I would try some of the things above before, so that you can provide useful information

Please let us know about your progress.

Thanks

studzik5521
Posted (edited)

I found this in the syslogs

Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: device recovery successful
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: Uncorrectable (Non-Fatal) error message received from 0000:00:01.0
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Requester ID)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:   device [8086:6f02] error status/mask=00100000/00000000
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:    [20] UnsupReq               (First)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER:   TLP Header: 34000000 01000010 00000000 88468846
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: device recovery successful
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: Uncorrectable (Non-Fatal) error message received from 0000:00:01.0
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Requester ID)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:   device [8086:6f02] error status/mask=00100000/00000000
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:    [20] UnsupReq               (First)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER:   TLP Header: 34000000 01000010 00000000 90059005
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: device recovery successful
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: Uncorrectable (Non-Fatal) error message received from 0000:00:01.0
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Requester ID)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:   device [8086:6f02] error status/mask=00100000/00000000
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:    [20] UnsupReq               (First)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER:   TLP Header: 34000000 01000010 00000000 88468846
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: device recovery successful
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: Uncorrectable (Non-Fatal) error message received from 0000:00:01.0
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Requester ID)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:   device [8086:6f02] error status/mask=00100000/00000000
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0:    [20] UnsupReq               (First)
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER:   TLP Header: 34000000 01000010 00000000 90059005
Feb 04 12:04:27 eddy kernel: pcieport 0000:00:01.0: AER: device recovery successful

I found a solution on this page LINK, adding the option "pcie_aspm=off" to the kernel options helped. Unfortunately, this is a workaround.

Although the error does not occur, CUDA still does not work.

The oldest driver version I was able to install was 535.98, older versions threw an error when compiling the kernel module. Probably proxmox has too new a kernel for older drivers.

All these versions had the same problem: 535.183.01, 535.98, 550.107.02, 550.142, 560.35.03 and 565.77 but then I had a problem with the message in syslog. 

In the next few days I will check nvidia toolkit, other ffmpeg and I will also check emby logs to see if they have changed.

Edited by studzik5521
  • Like 1
Posted

Probably the pci error is not related to the Nvidia GPU.

On the Nviia site where you download the drivers, there's a link for each driver to a huge document which also has section like "Troubleshooting" and "Common Issues". It's a lot to ingest, but maybe something jumps in your eyes which could apply to your system.

Like here for driver version 565.77: https://us.download.nvidia.com/XFree86/Linux-x86_64/565.77/README/index.html

 

studzik5521
Posted

I guess it's a problem with proxmox or kernel.
Check it later because I don't have the energy to do it now 😅

  • Like 1
studzik5521
Posted

I don't know if I built the test command correctly, but it seems to work.

Please check it and correct me if necessary.
Thank you.

root§docker-portainer:~/nvidia-patch£ ffmpeg -y     -init_hw_device "cuda=cuda:0"     -f lavfi -i "testsrc=size=1280x720:rate=25,format=yuv420
p"     -f lavfi -i "sine=frequency=1000:sample_rate=44100"     -filter_complex "¡0:v¿hwupload_cuda,scale_cuda=640:360:format=yuv420p,setsar=sa
r=1¡f2_out0¿"     -map "¡f2_out0¿" -map 1:a     -c:v h264_nvenc -b:v 464k -g 75 -maxrate 464k -bufsize 928k -keyint_min 75 -r 25 -profile:v hi
gh     -c:a libmp3lame -b:a 192k -ac 2 -filter:a "volume=3" -metadata:s:a:0 language=eng     -max_delay 5000000 -avoid_negative_ts disabled   
  -f segment -segment_format mpegts -segment_list "test.m3u8"     -segment_list_type m3u8 -segment_time 3 -segment_start_number 0     "test_%d
.ts"
ffmpeg version 5.1.6-0+deb12u1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input £0, lavfi, from 'testsrc=size=1280x720:rate=25,format=yuv420p':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream £0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720 ¡SAR 1:1 DAR 16:9¿, 25 tbr, 25 tbn
Input £1, lavfi, from 'sine=frequency=1000:sample_rate=44100':
  Duration: N/A, start: 0.000000, bitrate: 705 kb/s
  Stream £1:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
Stream mapping:
  Stream £0:0 (rawvideo) -> hwupload_cuda:default (graph 0)
  setsar:default (graph 0) -> Stream £0:0 (h264_nvenc)
  Stream £1:0 -> £0:1 (pcm_s16le (native) -> mp3 (libmp3lame))
Press ¡q¿ to stop, ¡?¿ for help
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_0.ts' for writing
Output £0, segment, to 'test_%d.ts':
  Metadata:
    encoder         : Lavf59.27.100
  Stream £0:0: Video: h264 (High), cuda(progressive), 640x360 ¡SAR 1:1 DAR 16:9¿, q=2-31, 464 kb/s, 25 fps, 90k tbn
    Metadata:
      encoder         : Lavc59.37.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 464000/0/464000 buffer size: 928000 vbv_delay: N/A
  Stream £0:1(eng): Audio: mp3, 44100 Hz, stereo, fltp, 192 kb/s
    Metadata:
      encoder         : Lavc59.37.100 libmp3lame
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing/A speed=3.17x    
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_1.ts' for writing
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing/A speed=4.27x    
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_2.ts' for writing
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing/A speed=4.66x    
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_3.ts' for writing
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing/A speed=4.94x    
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_4.ts' for writing
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing/A speed=5.08x    
¡segment § 0x5ecc3f8e8e40¿ Opening 'test_5.ts' for writing
¡segment § 0x5ecc3f8e8e40¿ Opening 'test.m3u8.tmp' for writing
frame=  380 fps=133 q=15.0 Lsize=N/A time=00:00:15.04 bitrate=N/A speed=5.26x    
video:263kB audio:342kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...