HW acceleration in LXC on proxmox

[LeidenSpain 33]

Hello,

 

After these days of distraction, I have already been able to try again with a clean installation of proxmox and everything starting from 0.

From what I see, the machine with emby perfectly recognizes the graphics (model and features), but I think it doesn't have permission to use them.

 

This are the permission from the folder /dev/dri on the host

root@server:~# ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root         80 Dec 27 20:25 by-path
crw-rw---- 1 root video  226,   0 Dec 27 20:25 card0
crw-rw---- 1 root render 226, 128 Dec 27 20:25 renderD128

 

And now, from the CT:

root@emby:~# ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root         80 Dec 27 19:25 by-path
crw-rw---- 1 root video  226,   0 Dec 27 19:25 card0
crw-rw---- 1 root syslog 226, 128 Dec 27 19:25 renderD128

 

 

hardware_detection-63807765955.txt

Edited December 29, 2022 by LeidenSpain

[Luke 42077]

HI, perhaps you need to configure Proxmox to pass through the gpu to Emby?

[LeidenSpain 33]

1 hour ago, Luke said:

HI, perhaps you need to configure Proxmox to pass through the gpu to Emby?

Hi,

Quite possibly, but theoretically I have it all right, but it doesn't work.
The machine I'm using is a CT (LXC Container), and the setup already allows you to access the graph.

As it is not a VM, it is not necessary to perform a pass through (I have it on another computer and with a Windows VM, to speed up the analysis of security cameras).

 

In the emby container I have transferred the graph, and if I look at hardware_detection or vainfo, it recognizes the graph correctly (so I understand that access to the device is fine).
The problem seems to be with some permission, or something with the DRM

[LeidenSpain 33]

Hi,

 

Yesterday I finally got the tip that has allowed the emby LXC to work perfectly.
Indeed, I have solved the problem by modifying the permissions of the /dev/dri/ folder on the host machine.

chmod 666 /dev/dri/*

 

This change is removed when the host machine is rebooted, so I will have to add it to cron so that it is modified at each start.

 

Thanks for all the help, now I have to talk to the developer of the emby script to put it in the documentation.

Edited December 30, 2022 by LeidenSpain

[Luke 42077]

Thanks for the feedback !

[LeidenSpain 33]

3 hours ago, Luke said:

Thanks for the feedback !

To you for all the times you help the community.
For once I discover the solution to a problem, it's only fair to share it with everyone.

[scyto 0]

Interesting thread.

On my system emby user in the LXC container (CT)  is assigned to the video and render groups.

These groups are mapped to the host groups render and video.

This means emby absolutely should have access to the following:

root@vGPU-debian-test:~# groups emby
emby : emby video render

root@vGPU-debian-test:~# ls -ls /dev/dri
total 0
0 crw-rw---- 1 nobody video  226,   6 Oct  4 21:42 card0
0 crw-rw---- 1 nobody render 226, 134 Oct  4 21:42 renderD128

(root in the CT has the same access)

with this level of permissions vainfo runs just fine, however something like intel_gpu_top doesn't

the reason for that is intel_gpu_top doesn't actually seem to use the the /dev/dri path for all access - as such it sees all 7 VFs on the host.

emby seems to do the same thing for it's detection logic (i suspect it is calling the i915 driver directly or maybe more...) sees all 7 host VFs instead of the CT VFs - this als means it doesn't understand that the /dev/dri/renderD134 (and card6) is actually mapped to renderD128 (and card0) in the CT.

I should add that when the container is privileged and used VFs the chmod 666 fix does not work.

 

is there a way to force /dev/dri/render128 and bypass the detection logic? 

(assuming that emby only uses that path when running...)

 

 

Edited October 5, 2023 by scyto

[LeidenSpain 33]

2 hours ago, scyto said:

Interesting thread.

On my system emby user in the LXC container (CT)  is assigned to the video and render groups.

These groups are mapped to the host groups render and video.

This means emby absolutely should have access to the following:

root@vGPU-debian-test:~# groups emby
emby : emby video render

root@vGPU-debian-test:~# ls -ls /dev/dri
total 0
0 crw-rw---- 1 nobody video  226,   6 Oct  4 21:42 card0
0 crw-rw---- 1 nobody render 226, 134 Oct  4 21:42 renderD128

(root in the CT has the same access)

with this level of permissions vainfo runs just fine, however something like intel_gpu_top doesn't

the reason for that is intel_gpu_top doesn't actually seem to use the the /dev/dri path for all access - as such it sees all 7 VFs on the host.

emby seems to do the same thing for it's detection logic (i suspect it is calling the i915 driver directly or maybe more...) sees all 7 host VFs instead of the CT VFs - this als means it doesn't understand that the /dev/dri/renderD134 (and card6) is actually mapped to renderD128 (and card0) in the CT.

I should add that when the container is privileged and used VFs the chmod 666 fix does not work.

 

is there a way to force /dev/dri/render128 and bypass the detection logic? 

(assuming that emby only uses that path when running...)

 

 

Finally I solved it permanently by adding this line to the conf file of the machine you are configuring:

lxc.hook.pre-start: sh -c "chown 0:107 /dev/dri/renderD128"

 

With this I haven't had to touch anything else and it works perfectly for me.

[scyto 0]

11 hours ago, LeidenSpain said:

lxc.hook.pre-start: sh -c "chown 0:107 /dev/dri/renderD128"

Thanks, and that script is executed in the container right?

(i have proven other simillar apps to emby can use /dev/dri paths with no issues - this is definitely an emby specific thing to do with how they validate hardware detection)

Edited October 5, 2023 by scyto

[scyto 0]

On 10/5/2023 at 1:52 AM, LeidenSpain said:

lxc.hook.pre-start: sh -c "chown 0:107 /dev/dri/renderD128"

I managed to test this today, i tried the gid of both the host render group (104) and the guest render group (106) -  neither helped

The permissions fix i had already put in place has given me the right permissions with no need to chown the host.

proxmox-vGPU.md (github.com)

[Schleudertrauma 10]

If you wan't to passthrough hardware into an lxc in proxmox the lxc needs to be privileged.

Otherwise one is running into several permission issues including accessing the host hardware because proxmox points the root user from the inside of an unprivileged container to a random user on the host system so that even if the lxc is compromised it is not possible to gain root access on the host system through the unprivileged lxc.

With easy words: In Proxmox the root user in an unprivileged container has NO root access on the host.