Jump to content

Enable hardware acceleration on FreeBSD


Duffyx

Recommended Posts

bingemeister
19 hours ago, softworkz said:

One more thing: Could you perhaps re-tar your modified folder tree and post it here for other users?

Sure, the files are attached.

11 hours ago, Duffyx said:

Have you gotten the link working in a jail?

Yep, I have it working in a jail. I just had to do the linking from the host environment because it wouldn't be allowed from within the jail. Your automatic solution looks even better! Just out of curiosity: Are you using iocage? I am still using ezjail and have thought about switching to iocage for a long time.

sys.tar

  • Like 1
Link to post
Share on other sites
8 minutes ago, bingemeister said:

Are you using iocage? I am still using ezjail and have thought about switching to iocage for a long time.

I don't use any front-end, I manually install a new jail with a "make installworld DESTDIR=/usr/jail/whateverjail" from the host and manage them manually with /etc/jail.conf :-).

Edited by Duffyx
Link to post
Share on other sites
10 hours ago, softworkz said:

So it is working for you as well now?

Yes, it is working fine now, thanks! 🥳
I hope we can get a proper solution in the future though.

  • Like 1
Link to post
Share on other sites

When it can just be installed and work is when I will use it again. Or at least some detailed step by step instructions. I am very Linux challenged.

Link to post
Share on other sites
bingemeister
11 minutes ago, dcol said:

I am very Linux challenged.

Well then I am happy to tell you that FreeBSD/FreeNAS is not Linux. 😀

They are both Unix-like but still quite different. And that‘s the reason why we are having this issue. Hardware acceleration works fine with Linux, but not on FreeBSD, because its different implementation is not handled correctly by Emby currently. But a solution is on the horizon now.

Link to post
Share on other sites

Linux, Unix, You get my point.

11 minutes ago, bingemeister said:

Well then I am happy to tell you that FreeBSD/FreeNAS is not Linux. 😀

They are both Unix-like but still quite different. And that‘s the reason why we are having this issue. Hardware acceleration works fine with Linux, but not on FreeBSD, because its different implementation is not handled correctly by Emby currently. But a solution is on the horizon now.

 

Link to post
Share on other sites
On 10/31/2020 at 4:36 AM, Duffyx said:

Have you gotten the link working in a jail? Or is this on the host? Could you share your devfs.rules :)?

NVM: In the meantime I found what's needed to create the symlink in the jail: 


exec.poststart = "cd /usr/jail/emby/dev/dri; ln -s card0 render0";

 

So wanting to use this in a TrueNAS 12 iocage jail I went looking for /dev/dri and it doesn't exist. There is a /dev directory in the jail but no /dri directory within it nor a /pci directory within /dev. 

Any clues for the more ordinary of us?

Link to post
Share on other sites
9 hours ago, Baenwort said:

So wanting to use this in a TrueNAS 12 iocage jail I went looking for /dev/dri and it doesn't exist. There is a /dev directory in the jail but no /dri directory within it nor a /pci directory within /dev. 

Any clues for the more ordinary of us?

The /dev/pci doesn't matter for now.

Do you have a /dev/drm directory in the jail? /dev/dri are just symlinks to /dev/drm.
If both do not exist you need to alter your devfs ruleset for that particular jail to "reveal" those device nodes in the jail.

Edited by Duffyx
  • Like 1
Link to post
Share on other sites
On 11/9/2020 at 7:58 AM, Duffyx said:

The /dev/pci doesn't matter for now.

Do you have a /dev/drm directory in the jail? /dev/dri are just symlinks to /dev/drm.
If both do not exist you need to alter your devfs ruleset for that particular jail to "reveal" those device nodes in the jail.

Here is what I have in Dev:

image.png.0fdbddfec28e8d66f7618716de31e8d6.png

This is inside a iocage managed jail on a TrueNAS Core 12 with the jail running the 12.1_RELEASE-p10 installed via the default FreeNAS procedure (not a plugin).

I have already added the mount some things.

image.thumb.png.6f7e24729d98e7fe4185fe8deb8e3f09.png

After work I'll dig into this more.

 

Link to post
Share on other sites
20 hours ago, Baenwort said:

Here is what I have in Dev:

image.png.0fdbddfec28e8d66f7618716de31e8d6.png

This is inside a iocage managed jail on a TrueNAS Core 12 with the jail running the 12.1_RELEASE-p10 installed via the default FreeNAS procedure (not a plugin).

I have already added the mount some things.

After work I'll dig into this more.

You need to create a new devfs ruleset that exposes the /dev/dri and /dev/drm of the host in the jail.

[devfsrules_drm=10]
add include $devfsrules_jail
add path dri unhide
add path dri/* unhide
add path drm unhide
add path drm/* unhide

This is at /etc/devfs.rules, then you point the jail to use number 10 devfs ruleset. Obviously this assumes you have the drm driver loaded on the host for your iGPU.

Edited by Duffyx
  • Like 1
Link to post
Share on other sites
7 hours ago, Duffyx said:

You need to create a new devfs ruleset that exposes the /dev/dri and /dev/drm of the host in the jail.


[devfsrules_drm=10]
add include $devfsrules_jail
add path dri unhide
add path dri/* unhide
add path drm unhide
add path drm/* unhide

This is at /etc/devfs.rules, then you point the jail to use number 10 devfs ruleset. Obviously this assumes you have the drm driver loaded on the host for your iGPU.

Does this look good? 

 

image.png.69ccaa9921e3115daf007e74dc983809.png

I already had a rule #10 and I've installed the intel drm drivers.

 

Link to post
Share on other sites

On FreeNAS/TrueNAS the devfs.rules resets after everyboot.

 

i've made a poststart_devfs.sh on my host and put in it:

#!/bin/sh

cat >>/etc/devfs.rules << 'EOF'

[devfsrules_bpfjail=101]
add path 'bpf*' unhide

[emby_drm=5]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_jail
add include $devfsrules_bpfjail
add path 'dri*' unhide
add path 'dri/*' unhide
add path 'drm*' unhide
add path 'drm/*' unhide

EOF


service devfs restart

I then set it as a postinit task via the TrueNAS gui.

However, I am not seeing a /dev/dri or /dev/drm yet.

Edited by Baenwort
Link to post
Share on other sites

Additional steps needed for TrueNAS 12 Core use within a jail:

From the main shell:

***Install multimedia/drm-kmod:

pkg install drm-kmod

***Enable loading of kernel module on boot: 

sysrc kld_list+="drm"

***Load the kernel module now: 

kldload drm

From the jail shell:

*** Install the supporting Intel VA support library for your GPU
*** libva-intel-driver: [LEGACY] Intel GMA 4500 or newer
*** libva-intel-media-driver: Intel HD 5000 (Gen8) or newer

pkg install libva-intel-media-driver

***Add emby user to the video group: 

pw groupmod -n video -m emby

 

Link to post
Share on other sites
2 hours ago, Baenwort said:

I now have the /drm folder in the jail

 

image.png.f110a68dded206b03d02809c61ba6192.png

but Emby does not yet see the intel GPU for acceleration :( 

so the fix for this was to put the following tunable in via the TrueNAS GUI

 

image.png.8660bcd093902bb0a5ff1d44889f1d4d.png

and to not need a reboot do enter this command in the main shell 

main shell:

kldload /boot/modules/i915kms.ko

I now have /dri in the jail:

image.png.dc93e2a42879ff788800632a0feb7d6b.png

However, still no HW accel showing in Emby in the jail. 

image.png

Edited by Baenwort
Link to post
Share on other sites
12 hours ago, Baenwort said:

On FreeNAS/TrueNAS the devfs.rules resets after everyboot.

This is weird, why would TruenNAS delete a file at random? The default devfs rules are at /etc/default/devfs.rules, the ones at /etc/devfs.rules should not be touched by the OS.

7 hours ago, Baenwort said:

so the fix for this was to put the following tunable in via the TrueNAS GUI
However, still no HW accel showing in Emby in the jail. 

You still need to apply the hack shown above; download the tarball by bingemeister and extract it in the root of the jail.
And also the symlink we spoke about (which I solved by the poststart="ln -s blabla").
The symlinking needs to happen on the host (wont work in the jail, not allowed).

The easiest way to verify you have done everything right so far is to run "vainfo" in the jail as the Emby user.
Vainfo can be found in the "libva-utils" package. The output should be similar as my post from 1 March 2020.

Edited by Duffyx
  • Like 1
Link to post
Share on other sites
17 hours ago, Duffyx said:

This is weird, why would TruenNAS delete a file at random? The default devfs rules are at /etc/default/devfs.rules, the ones at /etc/devfs.rules should not be touched by the OS.

You still need to apply the hack shown above; download the tarball by bingemeister and extract it in the root of the jail.
And also the symlink we spoke about (which I solved by the poststart="ln -s blabla").
The symlinking needs to happen on the host (wont work in the jail, not allowed).

The easiest way to verify you have done everything right so far is to run "vainfo" in the jail as the Emby user.
Vainfo can be found in the "libva-utils" package. The output should be similar as my post from 1 March 2020.

I agree but it seems to be true and if you search the FreeNAS/TrueNAS forums you'll see mentions of it by several other projects that are trying to pass things into jails.

I made a post-init command of "cd /grandmama/iocage/jails/emby/dev/dri; ln -s card0 render0" (my jail location) in the TrueNAS GUI.

image.thumb.png.31a237b4edf591ae1d8130c9403db33d.png

I've extracted the tar linked above to the root of the emby jail:

image.png.fd98846ced9cc742e724bec8a18954f8.png

and restarted the system.

And still no HW transcoding.

image.png.5c2a3364dba7bec105040e04c380aa0e.png

I'll resume troubleshooting another day.  Thank you for the help so far!

Link to post
Share on other sites

This is the last thing you are missing I think. Do you have a card0 entry and a symlink to card0?

Edited by Duffyx
Link to post
Share on other sites
On 11/12/2020 at 7:16 AM, Baenwort said:

I've extracted the tar linked above to the root of the emby jail:

I'll resume troubleshooting another day.  Thank you for the help so far!

I think you are missing this symlink:

 image.png.bbe3e8bf3ce7199ac766d17442655b6d.png

You should probably also verify if the post init script works and it creates the symlink in the devfs system in the jail.
Also as a next step, try running ffdetect manually at /usr/local/lib/emby-server/bin/ffdetect by using, from within the jail:

cd /usr/local/lib/emby-server/bin
su -m emby -c "./ffdetect vaenc"

 

Edited by Duffyx
  • Like 1
Link to post
Share on other sites
On 11/19/2020 at 2:44 AM, Duffyx said:

I think you are missing this symlink:

 image.png.bbe3e8bf3ce7199ac766d17442655b6d.png

You should probably also verify if the post init script works and it creates the symlink in the devfs system in the jail.
Also as a next step, try running ffdetect manually at /usr/local/lib/emby-server/bin/ffdetect by using, from within the jail:


cd /usr/local/lib/emby-server/bin
su -m emby -c "./ffdetect vaenc"

 

You are correct I am missing the card0.

image.png.ff88e8d9a94b6a5e367da215368410c3.png

I do have card0 at:

image.png.6bfd44d6a0d2aed558eb6a564b9a2c07.png

and here is the ffdetect:

image.thumb.png.5f2f4190018e2f7b42ecd288b5794bd6.png

I fixed the postinit script to:

cd /mnt/grandmama/iocage/jails/emby_2/root/dev/dri; ln -s card0 render0; as you where correct. I had made an error (my emby jail is now emby_2 as I duplicated it for testing) but I had not placed /mnt and /root in the tree as is actually needed.

image.png.052da6e304c0ed1f8db3cc07f7591aed.png

With the new postinit and I will turn things over and see if it works now.

 

Link to post
Share on other sites

Here is the ffdetect after fixing the postinit and restarting the jail.

root@emby_2:/usr/local/lib/emby-server/bin # su -m emby -c "./ffdetect vaenc"
ffdetect version 4.3.0-emby_2020_05_23 Copyright (c) 2018-2019 softworkz for Emby LLC
  built with FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1)
  configuration: --cc=clang --prefix=/usr/home/embybuilder/x64_freebsd12/ffmpeg-x64_freebsd12/staging --disable-amf --disable-debug --disable-doc --disable-ffplay --disable-vdpau --disable-xlib --enable-fontconfig --enable-gnutls --enable-gpl --enable-iconv --enable-libass --enable-libdav1d --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libzvbi --enable-pic --enable-version3 --enable-vaapi --arch=x86_64 --enable-shared --disable-static
  WARNING: library configuration mismatch
  avutil      configuration: --prefix=/usr/local --mandir=/usr/local/man --datadir=/usr/local/share/ffmpeg --pkgconfigdir=/usr/local/libdata/pkgconfig --enable-shared --enable-pic --enable-gpl --enable-avresample --cc=cc --cxx=c++ --disable-alsa --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libaom --disable-libaribb24 --enable-asm --enable-libass --disable-libbs2b --disable-libcaca --disable-libcdio --disable-libcelt --disable-libcodec2 --enable-libdav1d --disable-libdavs2 --disable-libdc1394 --disable-debug --enable-htmlpages --disable-libdrm --enable-libfdk-aac --disable-libflite --enable-fontconfig --enable-libfreetype --enable-frei0r --disable-libfribidi --disable-gcrypt --disable-libglslang --disable-libgme --enable-gmp --enable-gnutls --enable-version3 --disable-libgsm --enable-iconv --disable-libilbc --disable-libjack --disable-libklvanc --disable-libkvazaar --disable-ladspa --enable-libmp3lame --enable-liblensfun --enable-libbluray --disable-librsvg --disable-librtmp --disable-libxml2 --disable-lto --disable-lv2 --disable-mbedtls --disable-libmfx --disable-libmodplug --disable-libmysofa --enable-nonfree --disable-openal --disable-opencl --enable-libopencv --disable-opengl --disable-libopenh264 --enable-libopenjpeg --disable-libopenmpt --disable-openssl --enable-optimizations --enable-libopus --disable-pocketsphinx --disable-libpulse --disable-librabbitmq --disable-librav1e --enable-runtime-cpudetect --disable-librubberband --disable-sdl2 --disable-libsmbclient --disable-libsnappy --disable-sndio --disable-libsoxr --disable-libspeex --disable-libsrt --disable-libssh --disable-libtensorflow --disable-libtesseract --enable-libtheora --disable-libtwolame --enable-libv4l2 --enable-vaapi --disable-vapoursynth --enable-vdpau --disable-libvidstab --disable-libvmaf --enable-libvorbis --disable-libvo-amrwbenc --enable-libvpx --disable-vulkan --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --disable-libxavs2 --disable-libxcb --enable-libxvid --disable-outdev=xv --disable-libzimg --disable-libzmq --disable-libzvbi
  libavutil      56. 36.100 / 56. 51.100
Failed to open device at index 0: /sys/bus/pci/devices/._pci0000:00/class
GetDeviceName: unable to open pci.ids file
GetDeviceName unable to load pci.ids file
[DEVICE]
DeviceIndex=0
DEVICEINFO:VendorId=4660
DEVICEINFO:DeviceId=4369
DEVICEINFO:SubsytemVendorId=6900
DEVICEINFO:SubsytemDeviceId=4352
DEVICEINFO:DevPath=/sys/bus/pci/devices/pci0000:00
DEVICEINFO:DrmRender=/dev/dri/render0
DEVICEINFO:IsEnabled=1
DEVICEINFO:IsBootVga=1
DEVICEINFO:ERROR:Number=-1
DEVICEINFO:ERROR:Message=Failed to open the drm device /dev/dri/render0
[/DEVICE]

Still no card0 in jail:

image.png.1584c68322313616dbd02e169d37d096.png

but it is there outside the jail:

image.png.533bbcd4fd0cf86980784814da747967.png

Retiring to troubleshoot another day.  Thank you again for helping me see where I made mistakes!

image.png

Edited by Baenwort
Link to post
Share on other sites

You are confusing two directories let me explain, from within the jail you need to see the bottom list of files in /dev/dri.
You can create the symlink of render0 to card0 with the post init script as I showed. These are the hardware nodes ffmpeg uses for VAAPI.

root@emby:/dev/dri # ls -l
lrwxr-xr-x  1 root  wheel   8 Nov 16 16:17 card0 -> ../drm/0
lrwxr-xr-x  1 root  wheel   5 Nov 16 16:17 render0 -> card0
lrwxr-xr-x  1 root  wheel  10 Nov 16 16:17 renderD128 -> ../drm/128

Then we have the directory that takes care of hackishly feeding the devices in ffdetect for detection. Here you can create the symlink once, from within the jail. It does not need to be created every time because this is written on the local filesystem and is not part of devfs like the render node above. I also noticed you had weird permissions on the files extracted (group: staff?).

root@emby:/sys/bus/pci/devices/pci0000_00/drm # ls -l
-rwxr-xr-x  1 root  wheel   9 Oct 31 13:15 card0
lrwxr-xr-x  1 root  wheel   5 Oct 31 13:53 render0 -> card0

For the pci-ids error you need the following (install the package pciids, misc/pciids), we need to emulate the linux location of the pci.ids file.

root@emby:/usr/share/hwdata # ls -l
lrwxr-xr-x   1 root  wheel  31 Oct 31 10:18 pci.ids -> /usr/local/share/pciids/pci.ids

EDIT: I reuploaded the tar archive to this post.

sys.xz

Edited by Duffyx
  • Like 1
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...