Docker, SMB volume, and backup

[chops3246 4]

So I'm trying without much success just yet to get backup working on Emby. Running on Docker and I'm successfully using several read-only mounts for my various libraries that are located on my NAS. I was attempting to store emby backups on my NAS as well with the backup plugin, but I'm having trouble with volume permissions and am not sure how to solve it.

I've attached both a log and a sample of what my docker-compose.yml has for reference.

emby-backup near the end of that yml is the volume in question for backup storage on my NAS.

the log file shows the following error:

System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path '/mnt/emby-backup/embyserver-backup-full' is denied.

I'm sure this error is simply a file permissions issue, as a terminal inside the container shows the following for the process user and the /mnt/emby-backup permissions - I'm just needing a tip on how best to address this.

/ # ps -a
PID   USER     TIME  COMMAND
    1 root      0:00 s6-svscan -t0 /var/run/s6/services
   36 root      0:00 s6-supervise s6-fdholderd
  176 root      0:00 s6-supervise emby-server
  179 root      0:00 sh ./run
  199 bin       0:24 /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
  249 root      0:00 /bin/sh
  458 root      0:00 ps -a
/ # ls -lah /mnt/
total 8K     
drwxr-xr-x   12 root     root        4.0K Apr  1 01:50 .
drwxr-xr-x    1 root     root        4.0K Apr  1 01:50 ..
drwxr-xr-x    2 root     root           0 Mar 15 04:04 audio-books
drwxr-xr-x    2 root     root           0 Dec 24 02:09 common-photos
drwxr-xr-x    2 root     root           0 Apr  1 01:36 emby-backup
drwxr-xr-x    2 root     root           0 Mar 25 02:55 movies
drwxr-xr-x    2 root     root           0 Mar 31 20:36 music
drwxr-xr-x    2 root     root           0 Mar 30 05:21 photos-c
drwxr-xr-x    2 root     root           0 Mar 30 04:33 photos-j
drwxr-xr-x    2 root     root           0 Mar 26 02:14 tv
drwxr-xr-x    2 root     root           0 Feb 23 04:46 vidbox
drwxr-xr-x    2 root     root           0 Mar 17 03:18 videos-misc

 

embyserver.txt example.yml

[Lessaj 269]

I could be mistaken but your emby process runs as bin however the folder is owned by root with 755 permissions, so bin would not have access to write there. Changer the owner to bin and see if that works.

[chops3246 4]

Thanks - yes, indeed permissions issue. Before I posted, I had popped into a terminal for this and tried both a chmod 777 /mnt/emby-backup (w/o owner change) and a chown bin /mnt/emby-backup. Neither seemed to actually change the permissions nor the owner. I'm guessing that there's something up here with the volume setup using type cifs and such for the docker-compose.

I had even tried adjusting file and dir modes per docker docs on smb shares mounted as volumes per: https://docs.docker.com/storage/volumes/#create-cifssamba-volumes - also without any change in permissions that i could detect.

o: "username=myuser,password=mypassword,vers=3,sec=ntlmsspi,file_mode=0777,dir_mode=0777"

I must be missing something else. 

[Luke 39667]

Let us know if you find anything else. Thanks.

[chops3246 4]

Ah...well...I was still looking for a tip in case anyone sees something I'm missing. I'm kind of at the end of my rope for this backup permission issue (like I mentioned I'm not able to resolve it in the container directly by running the obvious commands). I was hoping someone else got this working and could share what they did.

Otherwise, I'd be using a local volume (not on a share that is backed up, dumping the cifs mount for this) and then do a song-and-dance like https://docs.docker.com/storage/volumes/#back-up-restore-or-migrate-data-volumes so I could be sure to have the emby backup file apart from the computer running the container. Docker's mount-volume-to-another-container-for-backup is less than ideal here, and I'd prefer to use the emby backup tool to send it to my NAS so it gets wrapped in with the rest of my backup strategy.

[Q-Droid 856]

Are you mounting the share from fstab? You can add uid= and gid= to the mount options to match the emby container uid/gid.

 

[chops3246 4]

Not quite. Like my yml shows above (and summarized below), I'm using this cifs volume docker feature to mount the share (had trouble doing this inside the container (as that's not as portable and requires extra tools). i've tried with and without the file_mode and dir_mode options. they don't seem to affect anything AFAICT.

 

emby-backup:

    driver: local

    driver_opts:

      device: "//192.168.0.80/toad/Container-Backups/Emby"

      type: "cifs"

      # found reference for file and dir modes here: https://docs.docker.com/storage/volumes/#create-cifssamba-volumes

      # originally i had issues with backups failing to my NAS drive due to permissions issues (even though the volume was writable, `bin` user for emby couldn't write there.

      o: "username=myuser,password=mypassword,vers=3,sec=ntlmsspi,file_mode=0777,dir_mode=0777"

 

[darkassassin07 604]

Why do you want/need the samba share mounted directly to the container?

 

I've got mine mounted to the host os, then used a bind mount to mount that to the container.

[chops3246 4]

also, i'm not clear on these settings and how they might affect emby execution or volume access (esp. since this is docker on windows and a smb/cifs share that doesn't directly support linux file permissions). per my docker compose yaml, i have these commented out. emby seems to run as "bin" when you are in the container terminal and do a ps -a.

 

# environment:

      # - UID=1000 # The UID to run emby as (default: 2)

      # - GID=100 # The GID to run emby as (default 2)

      # - GIDLIST=100 # A comma-separated list of additional GIDs to run emby as (default: 2)

[chops3246 4]

On 4/2/2024 at 9:51 PM, darkassassin07 said:

Why do you want/need the samba share mounted directly to the container?

 

I've got mine mounted to the host os, then used a bind mount to mount that to the container.

TBH this was about making it isolated from the underlying host os as much as possible to keep things more flexible. This way I'm not reliant on a share mapped on the host os, and it is a built-in docker feature for using a smb share as a volume. I thought it was quite handy for most things. Just having trouble with writing to it.

[chops3246 4]

So....is there anyone that has done something like this? I am still surprised there really isn't anything documentation or other threads on the web that I can line up here on this permissions issue.

[darkassassin07 604]

This seems to be an issue with docker and their implementation of samba mounting, it's not really embys wheelhouse.

Doesn't seem many people here use that function of docker. As I said, I just mount samba to the host, then bind mount the container; probably the more common method.

 

You'd might have better luck getting help with this specific issue on dockers forums.

[Luke 39667]

Can you mount the smb share in the os?

[chops3246 4]

OK I sorted it out. Came back around to just looking at the cifs man page - https://linux.die.net/man/8/mount.cifs.
My best estimate was this is what is used under the hood for docker desktop for the cifs volume type.

Realized from that the option I really need to care about here are `uid` and `gid` which were not present in any example I could find about cifs for docker in general.

Then was realizing the changes I was trying might not be taking effect as the volume was persistent...So I was essentially missing the proper parameters and I was testing this change incorrectly (i needed to stop and remove the container and then drop the backup volume and do an 'up' to put the compose all back in order with the new volume setting).

I created a powershell script just to run through the steps needed to make the mount change effective in the docker-compose. This allowed for faster iteration to find out what the issue was.

#stop container
docker-compose stop
#remove stopped container (to remove named volume)
docker-compose rm -f
#drop volume
docker volume rm emby_emby-backup
#recreate container and volume with new options
docker-compose up -d
#show us the permissions on the mount area
docker exec embyserver ls -lah /mnt

Here's what was actually needed in the end for the volume config to make the write possible. I tried a number of config options, but these two are all that was needed.

  emby-backup:
    driver: local
    driver_opts:
      device: "//192.168.0.80/toad/Container-Backups/Emby"
      type: "cifs"
      o: "username=myuser,password=mypassword,vers=3,sec=ntlmsspi,uid=2,gid=2"
  emby-data:
    driver: local

Here's the container mount config for that volume (not much special here, omitted all but the backup mount):

volumes:
      - emby-backup:/mnt/emby-backup

How did I know to use user and group id of "2" for the cifs options? This below is in the docker compose template from emby docs. Because these are commented out, it will default as uid and gid 2 -> this is the 'bin' user.

# environment:
      # - UID=1000 # The UID to run emby as (default: 2)
      # - GID=100 # The GID to run emby as (default 2)
      # - GIDLIST=100 # A comma-separated list of additional GIDs to run emby as (default: 2)

You can confirm this user with a terminal into the container like below. Note the "bin" user column for /system/EmbyServer. This will confirm you are looking at the right process in this context (in case you commented the above lines and changed to a new UID/GID you could translate that for the cifs options).

> docker exec embyserver ps -a
PID   USER     TIME  COMMAND
    1 root      0:00 s6-svscan -t0 /var/run/s6/services
   33 root      0:00 s6-supervise s6-fdholderd
  172 root      0:00 s6-supervise emby-server
  175 root      0:00 sh ./run
  198 bin       1:20 /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
  570 root      0:00 ps -a

This will allow you to mount a share from your NAS right into the container (instead via the container host running docker desktop or similar), making the config a bit more portable. The point in general for involving a share for backups is to put them somewhere besides where the container is running (NAS and docker host are separate in my case).

The end effect of the /mnt area I'm using is like below. Most of my mounts are read-only by design (so emby can't mess with my raw file data on the NAS). emby-backup is the only one here that is owned by bin user and bin group.

> docker exec embyserver ls -lah /mnt
total 8K     
drwxr-xr-x   12 root     root        4.0K Apr  6 01:58 .
drwxr-xr-x    1 root     root        4.0K Apr  6 01:58 ..
drwxr-xr-x    2 root     root           0 Mar 15 04:04 audio-books
drwxr-xr-x    2 root     root           0 Dec 24 02:09 common-photos
drwxr-xr-x    2 bin      bin            0 Apr  6 01:59 emby-backup
drwxr-xr-x    2 root     root           0 Apr  4 15:49 movies
drwxr-xr-x    2 root     root           0 Mar 31 20:36 music
drwxr-xr-x    2 root     root           0 Apr  5 05:02 photos-c
drwxr-xr-x    2 root     root           0 Apr  4 03:52 photos-j
drwxr-xr-x    2 root     root           0 Apr  2 01:48 tv
drwxr-xr-x    2 root     root           0 Feb 23 04:46 vidbox
drwxr-xr-x    2 root     root           0 Mar 17 03:18 videos-misc

One other wrench that made this harder to troubleshoot and I didn't see mentioned anywhere on the interwebs -> chmod or chown commands were not able to change the mount. So once the volume is created, it is "stuck" in that cifs options configuration and won't change. You must use the uid and gid options when re-creating the volume. This is something to bear in mind for production-related aspects (e.g. this would equate to an outage potentially for a critical business system).

TBH, this does seem like something important for the backup plugin for emby. You want that volume you use for backups to be living somewhere else, and this is a reasonable way to accomplish that.

Lastly, if you are in a bind and simply want to clone (to backup) the config volume for emby (or any other containers) and are using Docker Desktop this will make it a snap: https://hub.docker.com/extensions/docker/volumes-backup-extension.

 

Edited April 6, 2024 by chops3246

[Luke 39667]

Thanks for following up !