Docker

[Edrock200 38]

On 11/12/2014 at 10:08 PM, Luke said:

It is already up to date with the recent stable release.

4.8.1 is the latest stable, but it's not up on the official repo

[zer0ish 11]

1 minute ago, Edrock200 said:

4.8.1 is the latest stable, but it's not up on the official repo

Showed for me and I have that version currently. 

[Edrock200 38]

23 minutes ago, zer0ish said:

Showed for me and I have that version currently. 

Yep, you are 100% right. Didn't catch a hard coded version in my deployment, which is why watchtower wasn't pulling. My bad.

[darkassassin07 423]

Fairly recently switched to docker and am pretty happy with it so far.

 

I originally had my emby container run by the root user, but after switching it to be run by a different user, I now get the error "Error response from daemon: unknown or invalid runtime name: nvidia"

How do I give a non root user access to the nvidea daemon/runtime for use in docker?

 

Nvidia drivers are installed as well as nvidia-container-toolkit. Hardware transcoding is functioning, as long as it was root that ran the docker compose up command...

 

 

Running Debian 12

[Q-Droid 643]

Can you share your run or compose?

[darkassassin07 423]

version: "2.3"
services:
  embybeta:
    image: emby/embyserver:beta
    container_name: embybeta
    runtime: nvidia # Expose NVIDIA GPUs
    environment:
      - UID=1000 # The UID to run emby as (default: 2)
      - GID=1000 # The GID to run emby as (default 2)
#      - UID=0 # The UID to run emby as (default: 2)
#      - GID=0 # The GID to run emby as (default 2)
      - GIDLIST=100 # A comma-separated list of additional GIDs to run emby as (default: 2)
    volumes:
      - /srv/embybeta/programdata:/config # Configuration directory
      - /mnt/Drobo/Media:/Media:ro # Media directory
      - /mnt/Backups/Emby_Beta:/Backups #backup directory
      - /mnt/Cache/Emby_Cache_Beta:/Cache #cache
      - /mnt/Cache/Transcoding_Temp_Beta:/Transcoding_temp #transcoding directory
#    ports:
#      - 8096:8096 # HTTP port
    devices:
      - /dev/dri:/dev/dri # VAAPI/NVDEC/NVENC render nodes
    networks:
      - arr
    restart: unless-stopped
networks:
  arr:
    external: true

 

 

 

If I leave the 'runtime' and 'devices' sections un-commented and run as non-root I get the above error. Comment them out and it starts, but no HW Acceleration.

 

Running the same compose via sudo runs fine; but then it's not displayed/managed in docker desktop and seems to use seprate 'arr' networks internally. (I have an nginx container handling all the host network>docker connections, through the Internal 'arr' network)

Edited March 3 by darkassassin07

[Q-Droid 643]

Hmmm. That would point to access and/or visibility. I guess the question is whether the problem is access to the render devices for UID/GID/GIDLIST, which by the way don't look quite right to me. Or visibilty of the nvidia runtime for the UID/GID 1000.

 

[darkassassin07 423]

UID/GID 1000 is my main user. I've removed GIDLIST from my compose.

 

 

I'm not sure how to troubleshoot visibility issues between that id and the runtime.

 

Linux is relatively new beast for me. Trying to learn and explore. I appreciate the help though

[Q-Droid 643]

In a typical setup you use the GIDLIST to include access to the video/graphics render devices (/dev/dri). It's a list of the IDs for the groups, usually video and render. But I don't know what is needed to make sure the emby user (or whichever user ID is 1000 on your system) is able to use/reference the nvidia runtime. So you can start with the GIDLIST but more might be needed.

There's also this thread which you might have seen already and is a bit long...

 

Edited March 3 by Q-Droid

[darkassassin07 423]

With some more digging I've found there's separate daemon configs for root/non-root docker. Adding the Nvidia runtime config to the non-root version (via docker desktop, I'm not sure where that is stored on disk), now the non-root containers can see the Nvidia runtime and stopped reporting that error.

 

Now I'm getting the error:  "Error response from daemon: error gathering device information while adding custom device "/dev/dri": no such file or directory"

Trying to figure out the required permissions, but linux permissions have always confused me :/

 

Here's  "ls -n /dev/dri"
total 0
drwxr-xr-x  2 0   0       80 Mar  3 13:40 by-path
crw-rw----+ 1 0  44 226,   0 Mar  3 13:40 card0
crw-rw----+ 1 0 105 226, 128 Mar  3 13:40 renderD128

 

I've added "- GIDLIST=1000,226" to my compose, but that hasn''t helped.

[Q-Droid 643]

24 minutes ago, darkassassin07 said:

Here's  "ls -n /dev/dri"
total 0
drwxr-xr-x  2 0   0       80 Mar  3 13:40 by-path
crw-rw----+ 1 0  44 226,   0 Mar  3 13:40 card0
crw-rw----+ 1 0 105 226, 128 Mar  3 13:40 renderD128

That should be GIDLIST=44,105.

4 hours ago, darkassassin07 said:

devices:
      - /dev/dri:/dev/dri # VAAPI/NVDEC/NVENC render nodes

The devices entry in the compose you posted looks correct. 

 

[darkassassin07 423]

Huh, I thought those were 2 groups: card0 in 44+226, renderD128 in 105+226.

 

Anyway, added 226,44,and105 to the list, but same result:

 

"Error response from daemon: error gathering device information while adding custom device "/dev/dri": no such file or directory"

 

The /dev/ and /dev/dri/ folders both have "drwxr-xr-x" permissions, so I don't understand why it's not visible.

Edited March 4 by darkassassin07

[Q-Droid 643]

No, if you run ls -l you'll see one device is owned by root:video and the other root:render. 226 is the device number.

Have you looked through the docker Nvidia HW thread? The answer is very likely to be in there somewhere. I wish I could be more help but I don't use Nvidia GPU so no way for me to try things.

[darkassassin07 423]

Well... I think I know what the issue is: docker desktop, you're required to specify what folders are allowed to be shared with containers, as well as actually specify the mounts in the compose file. I'd added everything I needed except /dev/dri/....

 

Thing is now I suddenly can't run ANY containers from non-root. Every single container I try, including the 'hello-world' test docker container, just infinitely says 'Container <name> Starting' upon running 'docker compose up -d' and then just continuously counts up. Left it for over 1k seconds, none will start. As they never start, I don't get errors or exit codes.

 

Every compose I've tried with sudo added, runs fine.

 

I don't understand what's changed. I've done nothing but modify compose files and run docker compose up/down.

 

 

 

Edit:

Well, re-installed docker desktop and am back to being able to run containers as non-root; but adding /dev/dri to the list of allowed directories hasn't fixed my original issue.

Edited March 5 by darkassassin07

[Q-Droid 643]

Sounds like you have a few things going on that weren't clear before in the thread.

1. Running compose as root vs non-root. I usually run the administrative stuff as root (equiv to sudo). When the containers are created they still run with the given UID/PID unless none are given then it's root. Or more accurately the dockerd/containerd runtime user.

2. Docker Desktop vs Docker Engine. There are differences and likely config needed for Desktop to behave more like Engine on bare metal.

So when you run as root vs non-root do you mean the CLI/UI for config and compose or setting the UID/PUID/GID/PGID for the containers to 0? If it's the first then I think you're fine. Some containers need to be root, such as reverse proxies if you're running on ports below 1024.

[darkassassin07 423]

I'm starting to realize #2 is a big factor to my troubles. When I'm talking about running as non root, I literally just mean running the docker compose command without sudo, which should be exactly the same, given the user is in the 'docker' user group; running the container under the existing root docker engine as whatever uid/gid given in the command/compose.

 

Because I have docker desktop installed though, it runs a second docker engine under the users namespace instead of the root one. That engine is more isolated and needs more setup work than I expected (I didn't even realize they were two engines).

 

I think I'm going to purge docker, setup just the standard engine without desktop; and see how that goes. I'm creating more problems than I need. Docker Desktop for Linux is fairly new afaik and doesn't have much documentation/help.

Edited March 5 by darkassassin07

[Savagegek 1]

A few days back I reconfigured emby container, to use the host network to avoid seeing docker subnet ip's for all clients.

However it seems that this change from docker network to host, has changed the transcoding behaviour and specific transcoding almost everything because of the auto bitrate setting reports way to low bitrate.

Anybody else encountered these issues? No big issue for me, will be switching to a dedicated machine in a few months to enable HW transcoding.

[Q-Droid 643]

1 hour ago, Savagegek said:

A few days back I reconfigured emby container, to use the host network to avoid seeing docker subnet ip's for all clients.

However it seems that this change from docker network to host, has changed the transcoding behaviour and specific transcoding almost everything because of the auto bitrate setting reports way to low bitrate.

Anybody else encountered these issues? No big issue for me, will be switching to a dedicated machine in a few months to enable HW transcoding.

Is it transcoding for local or remote clients? Did you have a different subnet defined in the Emby server network settings? I don't think auto-detection applies to LAN clients, only remote. Aside from the Emby server potentially treating local clients as remote the network itself should should not affect transcoding.

[darkassassin07 423]

https://hub.docker.com/r/emby/embyserver/tags

 

4.8.4.0 is out, but only as it's own tag. Can we get the 'latest' branch/tag updated?

[alucryd 216]

4 hours ago, darkassassin07 said:

https://hub.docker.com/r/emby/embyserver/tags

 

4.8.4.0 is out, but only as it's own tag. Can we get the 'latest' branch/tag updated?

Done

[scottpro 4]

Got the Docker update version 4.8.4.0 this morning and it looks like "Schedule Tasks" no longer works as before.  I can only start the tasks and not schedule them.  When I click on a task now it brings up a dialog box to start the task only.  There used to be an arrow on the right side of the task box but is no longer there.  I would include screen shots but I already rolled back to 4.8.3.0.

[Luke 37066]

1 hour ago, scottpro said:

Got the Docker update version 4.8.4.0 this morning and it looks like "Schedule Tasks" no longer works as before.  I can only start the tasks and not schedule them.  When I click on a task now it brings up a dialog box to start the task only.  There used to be an arrow on the right side of the task box but is no longer there.  I would include screen shots but I already rolled back to 4.8.3.0.

Sounds like you have the web app set to TV display mode.

[scottpro 4]

11 hours ago, Luke said:

Sounds like you have the web app set to TV display mode.

Yup, TV mode.  I upgraded again to 4.8.4.0 and selected "Auto" as the Display Mode and now I can use the scheduler for tasks as well as start button.
So is this a UI feature/restriction by design?

Web browser with Display Mode - Auto, Desktop, Mobile/Tablet : Scheduled Tasks allows schedule editing and start button
Web browser with Display Mode - TV : Scheduled Tasks does not allow schedule editing, only start button
Emby Theater for Windows : Scheduled Tasks does not allow schedule editing, only start button
Mobile/Tablet app : Scheduled Tasks allows schedule editing and start button

BTW, I use TV display mode in my web browser so I can see what it will look like in my theater room when using ET.

[Luke 37066]

10 minutes ago, scottpro said:

Yup, TV mode.  I upgraded again to 4.8.4.0 and selected "Auto" as the Display Mode and now I can use the scheduler for tasks as well as start button.
So is this a UI feature/restriction by design?

Web browser with Display Mode - Auto, Desktop, Mobile/Tablet : Scheduled Tasks allows schedule editing and start button
Web browser with Display Mode - TV : Scheduled Tasks does not allow schedule editing, only start button
Emby Theater for Windows : Scheduled Tasks does not allow schedule editing, only start button
Mobile/Tablet app : Scheduled Tasks allows schedule editing and start button

BTW, I use TV display mode in my web browser so I can see what it will look like in my theater room when using ET.

Hi, yes the functions need to be added to TV mode. Thanks.

[darkassassin07 423]

Can we please get the docker image releases updated at the same time as every other release?

 

Or, have docker based emby servers check docker hub for their updates? (regarding when to send 'update available' notifications)

 

I keep getting notifications for updates that still don't exist multiple days after receiving the notification. Docker releases always seem to lag behind by as much as a full week... It's getting quite frustrating being told to update but not being able to.