Docker HWA / Disk permission

[Nuitarius 22]

So after testing I have several issues.

And I cannot start any movie anymore

I hope someone can help me out

no access to metadata either

 

It seems the docker works, and emby works but that access to all my disks are gone even though nothing has changed.

The only thing I can think of is that I used sudo command to run the docker which is also new as I earlier had so many issues with running it without sudo. Is that an issue?

But I also have lost docker desktop functionality, so I am handicapped on how to do stuff as I have only ever used that as interface.

Anyone have an idea?

[Nuitarius 22]

I deleted all containers and tried to go back to my old docker setup without --gpu all

I can create it but it now goes into continuous running/restarting loop, so I cannot get it back up running.

Any assitance is appreciated.

[Nuitarius 22]

In docker desktop i get some logging errors

2024-07-10 20:38:20 chown: /config/metadata/people/Steven Yeun-tmdb-215055: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people/Steven Yeun-tmdb-215055: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people/Michael Wincott-tmdb-7486/folder.jpg: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people/Michael Wincott-tmdb-7486: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people/Michael Wincott-tmdb-7486: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people: Operation not permitted
2024-07-10 20:38:20 chown: /config/metadata/people: Operation not permitted

 

It seems some of my tinkering with installing the nvidia kits and driver has somehow removed som rights, so that the emby docker no longer can access files on my discs.

I can start the docker without --gpu all only if I use sudo in the run command now. That was not needed prior to today. And I can still not run movies and I cannot scan library. It still seems like the docker does not have access to disks anymore

 

Hmm. Maybe this nvidia thing has stopped docker desktop from working somehow. I know that I used Docker desktop to setup disk ressource


what if my setup somehow does not know about this setup anymore.

How would I get passed this? Is there a config file somewhere that needs to have this "file sharing" setting?

[Nuitarius 22]

I had a fully functioning docker, and then I started on my journey of adding a GPU and wanting to do HWA.

I have been far and wide and it seems for now that I got the HWA to work, but now I have a new problem. the emby docker cannot access my disks with movies. When choosing a movie in emby I get this feedback "No compatible streams are currently available. Please try again later or contact your system administrator for details."

I can view the mounted volumes inside the emby server (under Library where I can see them in the drop down), but I cannot add them as pressing on them does nothing (all other folders in the drop down can be added fine)
Before this HWA trip I ran docker with docker run command, without sudo. Now when wanting to use --gpus all command it seems I have to use sudo to have the run command actually execute successfully, else the command fails.

Without sudo:

docker run -d --volume /mnt/Disk1/EmbyServer/config:/config --volume /mnt/Disk1:/mnt/Disk1 --volume /mnt/Disk2:/mnt/Disk2 --volume /mnt/Disk3:/mnt/Disk3 --restart unless-stopped --gpus all --publish 8096:8096 --publish 8920:8920 --env UID=2 --env GID=2 --env GIDLIST=2 --name embyserv emby/embyserver:latest
cae0490e19cc9ce98b5e3e6c62f68f8830bf4f47ed4b90ef6fbcfeeabcd3fabd
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

With sudo:

sudo docker run -d --volume /mnt/Disk1/EmbyServer/config:/config --volume /mnt/Disk1:/mnt/Disk1 --volume /mnt/Disk2:/mnt/Disk2 --volume /mnt/Disk3:/mnt/Disk3 --restart unless-stopped --gpus all --publish 8096:8096 --publish 8920:8920 --env UID=2 --env GID=2 --env GIDLIST=2 --name embyserv emby/embyserver:latest
[sudo] password for nuitarius: 
c5d496f78d14eeba2460f3b180f8412c2e5b23de84217a808de078ac56128105

I am guessing that because I am running the docker with sudo that for some reason the docker cant access my mapped volumes. Is there anyone out there that can help a lad out

My new docker command line

sudo docker run -d --volume /mnt/Disk1/EmbyServer/config:/config --volume /mnt/Disk1:/mnt/Disk1 --volume /mnt/Disk2:/mnt/Disk2 --volume /mnt/Disk3:/mnt/Disk3 --restart unless-stopped --gpus all --publish 8096:8096 --publish 8920:8920 --env UID=2 --env GID=2 --env GIDLIST=2 --name embyserv emby/embyserver:latest

My old command line

docker run -d --volume /mnt/Disk1/EmbyServer/config:/config --volume /mnt/Disk1:/mnt/Disk1 --volume /mnt/Disk2:/mnt/Disk2 --volume /mnt/Disk3:/mnt/Disk3 --restart unless-stopped --publish 8096:8096 --publish 8920:8920 --env UID=2 --env GID=2 --env GIDLIST=2 --name embyserv emby/embyserver:latest

My old command line does not work anymore as the emby docker goes into restart loop. Why I do not know. The log file states a lot of permission failures:

2024-07-12 23:52:30 chown: /config/metadata/people/Daniel Kaluuya-tmdb-206919: Operation not permitted

I have not deliberately or knowingly changed anything about permissions, so I do not know why this should be an issue, but it sure looks like it is now

Edited July 12, 2024 by Nuitarius

[Luke 42077]

Hi there, can you please attach the emby server log? Thanks.

[Nuitarius 22]

2 hours ago, Luke said:

Hi there, can you please attach the emby server log? Thanks.

Can these be used? These are the newest ones, or else please let me know

I am currently running the sudo version with --gpus all

embyserver.txt embyserver-63856425610.txt hardware_detection-63856421843.txt

Edited July 13, 2024 by Nuitarius

[Nuitarius 22]

I am finding a lot of access to paths denied in the logs

System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path '/mnt/Disk1/EmbyServerMetadata/metadata/library/18/1876454d44d34951b8c30200286fbe60' is denied.
     ---> System.IO.IOException: Permission denied

My guess is that starting the docker with sudo command gets you this.

So there should be 2 solutions

1. How to start the docker without sudo, but still utilizing --gpus all?

2. How to get docker to accept the mounted volumes even using sudo?

 

I have actually always had issues with volumes now when thinking about it. I only get it to work if I add the mounted drives (e.g. /mnt/Disk1) in Docker Desktop under file sharing. If I do not do that the docker run command will not work.

When using sudo in my run command I also do not get to see the docker in Docker Desktop. As if it is not accessible for Docker Desktop. So that might also mean that the file sharing done in Docker Desktop will not work for dockers created by sudo.

So how do I go by and get the file sharing to work with the sudo command?

[Nuitarius 22]

@Lukehi. Have you had a chance to see the log?

 

On 7/13/2024 at 6:22 AM, Luke said:

Hi there, can you please attach the emby

 

 

[Luke 42077]

4 hours ago, Nuitarius said:

@Lukehi. Have you had a chance to see the log?

 

 

 

Hi, you configured a number of custom paths but it seems the server is being denied write access to those locations. This includes:

• The server metadata folder
• The backup plugin folder

If you can resolve this, then those issues should go away.  My suggestion for the server metadata folder would be to just use the default, so that's an easy fix there. Just make sure to read the help text underneath the option when changing it.

[Nuitarius 22]

20 hours ago, Luke said:

Hi, you configured a number of custom paths but it seems the server is being denied write access to those locations. This includes:

• The server metadata folder
• The backup plugin folder

If you can resolve this, then those issues should go away.  My suggestion for the server metadata folder would be to just use the default, so that's an easy fix there. Just make sure to read the help text underneath the option when changing it.

Thank you @LukeI do appreciate it.

Could you perhaps help a brother out here in how I can search for a reason for why I suddenly do not have disk permissions when running docker with sudo command?

It worked before I installed the Nvidia driver, so I am confused on what has changed.

[Luke 42077]

I don't know the answer to that. @Q-Droidmay have some tips though.

[Q-Droid 989]

@Nuitarius I can guess a few things.

Docker Desktop is a development environment that runs a virtual machine on the host for the containers to reside. It adds a layer that might explain why the GPU is not accessible to an unprivileged user.

UID/GID 2 very likely doesn't have access to the library paths. Chances are that it won't have access to the GPU either even if it can access the driver. When running without sudo my guess is that it ignored the UID/GID and ran as your login, nuitarius. Changing runtime user is a privileged operation and why it works with sudo. Going back and forth between sudo and non-sudo is going to create problems with Emby server file ownership.

A quick fix might be to continue to run using sudo. Change the UID, GID and GIDLIST to better values. Pick a UID/GID that has access to the media paths and files on the host and GIDLIST should contain the group values listed for the devices in /dev/dri. 

Long term:
 - get rid of Docker Desktop and install Docker Engine.
 - get comfortable with Docker compose files and/or Portainer if you want an UI.
 - (optional but prudent) create a user and group to use for the Emby server runtime. This new user/group will need to own or have access to the media paths.

 

[Nuitarius 22]

7 hours ago, Q-Droid said:

@Nuitarius I can guess a few things.

Docker Desktop is a development environment that runs a virtual machine on the host for the containers to reside. It adds a layer that might explain why the GPU is not accessible to an unprivileged user.

UID/GID 2 very likely doesn't have access to the library paths. Chances are that it won't have access to the GPU either even if it can access the driver. When running without sudo my guess is that it ignored the UID/GID and ran as your login, nuitarius. Changing runtime user is a privileged operation and why it works with sudo. Going back and forth between sudo and non-sudo is going to create problems with Emby server file ownership.

A quick fix might be to continue to run using sudo. Change the UID, GID and GIDLIST to better values. Pick a UID/GID that has access to the media paths and files on the host and GIDLIST should contain the group values listed for the devices in /dev/dri. 

Long term:
 - get rid of Docker Desktop and install Docker Engine.
 - get comfortable with Docker compose files and/or Portainer if you want an UI.
 - (optional but prudent) create a user and group to use for the Emby server runtime. This new user/group will need to own or have access to the media paths.

 

@Q-Droidthank you for helping out.

I will look into your suggestions and give feedback here when/if I got smarter and fixed it.

[Nuitarius 22]

@Q-Droid
Status:
I have now uninstalled everything about docker, re-installed docker-ce and installed compose and portainer.

Portainer was teasing me during install due to docker context was not set to default.

I have also got my docker run command without sudo to work incl. the "--gpu all"

Portainer works and says the emby server is running.

I can access the server incl logging in.

sudo goodbye
 

Sadly Emby still did not have access to any disks, so back to your excellent guestimation on my issues. I looked up the ID thing and finally understood what you meant (yeah I am a little thick).

My new run command goes:

docker run -d --volume /mnt/Disk1/EmbyServer/config:/config --volume /mnt/Disk1:/mnt/Disk1 --volume /mnt/Disk2:/mnt/Disk2 --volume /mnt/Disk3:/mnt/Disk3 --restart unless-stopped --gpus all --publish 8096:8096 --publish 8920:8920 --env UID=1000 --env GID=1000 --env GIDLIST=1000 --name embyserv emby/embyserver:latest

 

This works like a charm, my movies can be played back again, hurray.

It looks like I have access to gpu, but as you stated I may not if these IDs are not set correctly. For the moment I do not believe I have full access but I will have to test further.

Can you give me a hint on how to find the correct ID and which of the IDs needs to be set correctly for HWA to work?

And thank you again so dearly for helping me out.

[Q-Droid 989]

You want to get the host values for the video and render groups. Sometimes it's only video.

ls -l /dev/dri

getent group video | cut -d: -f3
getent group render | cut -d: -f3

GIDLIST=value,value

When using Docker a new container is created every time you execute the "docker run" command leaving inactive containers behind if they have the same image and name. Once you get a working container with the right settings you can then use docker container start/stop or the Portainer UI to manage them and declutter.

 

[Nuitarius 22]

@Q-DroidHmm... I am having some issues fully grasping what you want me to do here, but hopefully you can tell if I am getting close

As you can see beneath I for some reason have 2 video and 2 renders (I only have 1 Quadro card but one of them might be the onboard one?)

$ ls -l /dev/dri
total 0
drwxr-xr-x  2 root root        120 Jul 18 09:41 by-path
crw-rw----+ 1 root video  226,   0 Jul 18 09:41 card0
crw-rw----+ 1 root video  226,   1 Jul 18 09:41 card1
crw-rw----+ 1 root render 226, 128 Jul 18 09:41 renderD128
crw-rw----+ 1 root render 226, 129 Jul 18 09:41 renderD129
$ getent group video | cut -d: -f3
44
$ getent group render | cut -d: -f3
106

I therefore assumed that I have to change my "GIDLIST=1000" to "GIDLIST=44,106"

Is that a fair assumption, or do I need to do something to ensure that I am getting the ID from the "correct" card?

[Lessaj 467]

lspci | grep VGA

Will show you the VGA controllers detected. Also you can look at /dev/dri/by-path/ and it will have the PCI paths for the devices. In this example it's 0d:00.0 which is my AMD card.

crw-rw----+ 1 root video  226,   1 Jul 16 09:36 card1
crw-rw-rw-  1 root render 226, 128 Jul 16 09:36 renderD128

lrwxrwxrwx 1 root root  8 Jul 13 18:09 pci-0000:0d:00.0-card -> ../card1
lrwxrwxrwx 1 root root 13 Jul 13 18:09 pci-0000:0d:00.0-render -> ../renderD128

0d:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c1)

 

[Q-Droid 989]

17 minutes ago, Nuitarius said:

@Q-DroidHmm... I am having some issues fully grasping what you want me to do here, but hopefully you can tell if I am getting close

As you can see beneath I for some reason have 2 video and 2 renders (I only have 1 Quadro card but one of them might be the onboard one?)

$ ls -l /dev/dri
total 0
drwxr-xr-x  2 root root        120 Jul 18 09:41 by-path
crw-rw----+ 1 root video  226,   0 Jul 18 09:41 card0
crw-rw----+ 1 root video  226,   1 Jul 18 09:41 card1
crw-rw----+ 1 root render 226, 128 Jul 18 09:41 renderD128
crw-rw----+ 1 root render 226, 129 Jul 18 09:41 renderD129
$ getent group video | cut -d: -f3
44
$ getent group render | cut -d: -f3
106

I therefore assumed that I have to change my "GIDLIST=1000" to "GIDLIST=44,106"

Is that a fair assumption, or do I need to do something to ensure that I am getting the ID from the "correct" card?

Yes, you assign 44 and 106 to GIDLIST. 

This might get more things working. Intel HW accel is easy with Emby, Nvidia might take a little more effort. 

If the quadro card still doesn't work there are threads/posts with examples to get Nvidia HW working with Emby in docker containers. 

[Nuitarius 22]

I am running 2 transcoded streams at the same time and CPU has not been impacted. I would assume that is a good sign.

Can I assume that the Quadro card is in use then?

[Q-Droid 989]

Check the ffmpeg-transcode logs, they should include the decode/encode details for those sessions. 

Edit to add: in the emby server transcoding options you can change the enable hw accel  picklist to advanced and it will then list all of the detected graphics devices.

 

 

Edited July 18, 2024 by Q-Droid

[Lessaj 467]

Further to that if you have the Diagnostics Plugin you can see from the User Sessions tab what GPU is being used as well.

Edited July 18, 2024 by Lessaj

[Nuitarius 22]

It seems I have gotten through to the GPU. Eureka.

 

Thank you @Q-Droidand @Lessajfor your assistance. It has been helpful indeed.

>>>>>>  User policy for Transcode
        Enable Playback Remuxing: True
        Enable Video Playback Transcoding: True
        Enable Audio Playback Transcoding: True

>>>>>>  Hardware Decoders for hevc
        [X] NVDEC Quadro P1000 - H.265 (HEVC)
        [ ] CUVID Quadro P1000 - H.265 (HEVC)

>>>>>>  Hardware Encoders for h264
        [X] NVENC Quadro P1000 - H.264 (AVC)

>>>>>>  Selected Codecs
Decoder NVDEC Quadro P1000 - H.265 (HEVC)
        Adapter #0: 'Quadro P1000' ComputeCapability: 6.1
        Max Bitrate: 117 Mbit/s - Frame Sizes: 144x144...8192x8192 - Width Alignment: 2 - Height Alignment: 2
        Color Formats: NV12, P010, P016 - Bit Depths: 8, 10, 12, 14, 16
        Profiles: Main Profile (Level 6.1), Main 10 Profile (Level 6.1), Range Extension (REXT) (Level 6.1)

Encoder NVENC Quadro P1000 - H.264 (AVC)
        Adapter #0: 'Quadro P1000' ComputeCapability: 6.1
        Max Bitrate: 781 Mbit/s - Frame Sizes: max 4096x4096 - Width Alignment: 2 - Height Alignment: 2
        Color Formats: NV12, P010, P016 - Bit Depths: 8, 10, 12, 14, 16
        Profiles: Baseline Profile (Level 6.2), Main Profile (Level 6.2), High Profile (Level 6.2), High 4:4:4 Predictive Profile (Level 6.2)

>>>>>>  FindVideoEncoder - MediaType: h264, UseHardwareCodecs: True, HWA-Mode: Advanced
Info    Checking: 'NVENC Quadro P1000 - H.264 (AVC)'
Info    Check successful - selecting 'NVENC Quadro P1000 - H.264 (AVC)'

>>>>>>  FindVideoDecoder - MediaType: hevc, UseHardwareCodecs: True, HWA-Mode: Advanced
Info    Checking: 'NVDEC Quadro P1000 - H.265 (HEVC)'
Info    Check successful - selecting 'NVDEC Quadro P1000 - H.265 (HEVC)'
Info    Tone Mapping would be desired, but hardware tone mapping is disabled

>>>>>>  Processing Plan
        Name                                        CanDoInHW  WillDoInHW  Reason                                                 
        NVDEC Quadro P1000 - H.265 (HEVC)        >> True       True        Hardware Codec                                          
        VideoInput                               >> True       True        Matching hardware context                               
        Scaling                                  >> True       True                                                                
        ToneMapping (when possible)              >> True       True                                                                
        VideoOutput                              >> True       True        Hardware encoder                                        
        NVENC Quadro P1000 - H.264 (AVC)         >> True       True        Hardware Codec                                          

>>>>>>  Video Processing Steps for [0:0]: H.265 (HEVC)
        Step                    HW-Context   Format       SW-Format           Size   Next
        HEVC                 >> CUDA         cuda         yuv420p10      3840x2160 >> scale_cuda
        scale_cuda           >> CUDA         cuda         yuv420p        1920x1080 >> setsar
        setsar               >> CUDA         cuda         yuv420p        1920x1080 >> 

Edit:

I also did some testing. On my internal Wifi I got Wonder Woman in 4K originally 55mbps transcoded to 14mbps to a 4K Chromecast. During a half hour test it had no stutter/lag/audio delay or buffering.

It seems to work wonderfully

Edit:

I also enabled HW tone mapping, thanks to the log where it states that I should do that

Edited July 18, 2024 by Nuitarius