Jump to content

Docker thumbnail image extraction permission denied


Recommended Posts

billg

The Docker Emby instance I am running is struggling to create Roku thumbnails for all media libraries. All media libraries are mounted via NFS on the underlying host OS. The host OS has no issues reading/writing files on the NFS share.

Running Emby in Docker it is able to create the various media metadata and the poster.jpg files and read/write/remove files in /config.

 

The container runs as UID 21023 (media user UID outside the container) and GID 30502 (media group GID outside the container); set as environment variables to Docker (along with media paths):

[root@dockerhost ~]# docker ps | grep embyserver
1797bf5e5d0d   emby/embyserver:latest                                                         "/init"                  About an hour ago   Up About an hour        1900/udp, 8096/tcp, 7359/udp, 8920/tcp   emby
[root@dockerhost ~]#
[root@dockerhost ~]# docker exec -it emby /bin/ash
/ #
/ # cd /config
/config # ps auxfww |grep Emby
  194 21023     2:56 /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
 2143 root      0:00 grep Emby
/config # echo $UID
21023
/config # echo $GID
30502
/config #

 

Sample log when extracting thumbnails. Ffmpeg successfully populates the cache folder with images.

2021-04-27 12:53:02.094 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/media/tv/Series.2020/S01/Series.2020.S01E08.mkv" -an -sn -s 320x180 -vsync cfr -r 0.1 -f image2 "/config/cache/temp/2766f0b8faa944a9afab63c4bee591aa/img_%05d.jpg"
2021-04-27 12:53:02.095 Debug App: ProcessRun 'quick-extract-imageseries' Started.
2021-04-27 12:53:40.691 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 0
2021-04-27 12:53:41.586 Error ChapterImagesTask: Error creating thumbnails for /media/tv/Series.2020/S01/Series.2020.S01E08.mkv
	*** Error Report ***
	Version: 4.5.4.0
	Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
	Operating system: Linux version 5.11.11-arch1-1 (linux@archlinux) (gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP PREEMPT Tue, 30 Mar 2021 14:10:17 +0000
	Framework: .NET Core 3.1.8
	OS/Process: x64/x64
	Runtime: system/System.Private.CoreLib.dll
	Processor count: 8
	Data path: /config
	Application path: /system
	System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path is denied.
	 ---> System.IO.IOException: Operation not permitted
	   --- End of inner exception stack trace ---
	   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
	   at Interop.CheckIo(Int64 result, String path, Boolean isDirectory, Func`2 errorRewriter)
	   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken)
	Source: System.IO.FileSystem
	TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo])
	InnerException: System.IO.IOException: Operation not permitted
	Source: 
	TargetSite: 

2021-04-27 12:53:41.597 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/media/tv/Series.2020/S01/Series.2020.S01E09.mkv" -an -sn -s 320x180 -vsync cfr -r 0.1 -f image2 "/config/cache/temp/c0b04b9b1abf4dd995d2daede14ee5b9/img_%05d.jpg"
2021-04-27 12:53:41.599 Debug App: ProcessRun 'quick-extract-imageseries' Started.

 

In the container these are the permissions to the sample media path above:

/config # stat /media/
  File: /media/
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: b8h/184d	Inode: 7920        Links: 7
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-04-27 18:07:23.000000000
Modify: 2021-04-27 17:17:26.000000000
Change: 2021-04-27 17:17:26.000000000

/config #
/config # stat /media/tv/
  File: /media/tv/
  Size: 122       	Blocks: 129        IO Block: 131072 directory
Device: 42h/66d	Inode: 4           Links: 116
Access: (0775/drwxrwxr-x)  Uid: (20500/ UNKNOWN)   Gid: (30502/ UNKNOWN)
Access: 2016-12-13 16:20:00.000000000
Modify: 2021-04-26 23:30:09.000000000
Change: 2021-04-26 23:30:09.000000000

/config #
/config # stat /media/tv/Series.2020/
  File: /media/tv/Series.2020/
  Size: 10        	Blocks: 33         IO Block: 131072 directory
Device: 42h/66d	Inode: 164248      Links: 3
Access: (0775/drwxrwxr-x)  Uid: (21125/ UNKNOWN)   Gid: (30502/ UNKNOWN)
Access: 2020-03-30 00:40:24.000000000
Modify: 2020-03-30 13:51:54.000000000
Change: 2020-03-30 13:51:54.000000000

/config #
/config # stat /media/tv/Series.2020/S01/
  File: /media/tv/Series.2020/S01/
  Size: 33        	Blocks: 129        IO Block: 131072 directory
Device: 42h/66d	Inode: 164257      Links: 2
Access: (0775/drwxrwxr-x)  Uid: (21125/ UNKNOWN)   Gid: (30502/ UNKNOWN)
Access: 2020-03-30 00:55:00.000000000
Modify: 2021-04-27 17:53:41.000000000
Change: 2021-04-27 17:53:41.000000000

/config #
/config # stat /media/tv/Series.2020/S01/Series.2020.S01E08.mkv
  File: /media/tv/Series.2020/S01/Series.2020.S01E08.mkv
  Size: 1581685395	Blocks: 3086712    IO Block: 131072 regular file
Device: 42h/66d	Inode: 164254      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: (21125/ UNKNOWN)   Gid: (30502/ UNKNOWN)
Access: 2020-03-30 00:53:38.000000000
Modify: 2020-03-30 00:55:32.000000000
Change: 2020-03-30 01:09:50.000000000

/config #

 

Edited by billg
Link to post
Share on other sites
Luke

Hi, what about permissions on the parent folder of the video? It looks like the server is unable to copy files into there from the cache folder.

Link to post
Share on other sites
billg

Agreed it looks like a permission issue but, if so, how can the metadata refresh task write into the same folder? Are these tasks launched under different UIDs/GIDs internally?

 

Each directory's permission mode is always 775 from the volume mount point, /media/tv, all the way to the media. The UID may change depending on which user created the media file(s) however the GID is always 30502.

Contents of the series directory and permissions are:

/config # stat /media/tv/Series.2020/
  File: /media/tv/Series.2020/
  Size: 10        	Blocks: 33         IO Block: 131072 directory
Device: 42h/66d	Inode: 164248      Links: 3
Access: (0775/drwxrwxr-x)  Uid: (21125/ UNKNOWN)   Gid: (30502/ UNKNOWN)
Access: 2020-03-30 00:40:24.000000000
Modify: 2020-03-30 13:51:54.000000000
Change: 2020-03-30 13:51:54.000000000

/config # ls -al /media/tv/Series.2020/
total 1867
drwxrwxr-x    3 21125    30502           10 Mar 30  2020 .
drwxrwxr-x  116 20500    30502          122 Apr 26 23:30 ..
drwxrwxr-x    2 21125    30502           33 Apr 28 07:06 S01
-rwxrwxr-x    1 21023    30502        91371 Mar 30  2020 banner.jpg
-rwxrwxr-x    1 21023    30502       352199 Mar 30  2020 fanart.jpg
-rwxrwxr-x    1 21023    30502       293101 Mar 30  2020 landscape.jpg
-rwxrwxr-x    1 21023    30502        65896 Mar 30  2020 logo.png
-rwxrwxr-x    1 21023    30502       591428 Mar 30  2020 poster.jpg
-rwxrwxr-x    1 21023    30502       253743 Mar 30  2020 season01-poster.jpg
-rwxrwxr-x    1 21023    30502         3130 Mar 30  2020 tvshow.nfo
/config #
/config #
/config # ls -al /media/tv/Series.2020/S01/ | egrep 'season.nfo|S01E08'
-rwxrwxr-x    1 21023    30502       215751 Mar 30  2020 Series.2020.S01E08-thumb.jpg
-rwxrwxr-x    1 21125    30502    1581685395 Mar 30  2020 Series.2020.S01E08.mkv
-rwxrwxr-x    1 21023    30502         2397 Mar 30  2020 Series.2020.S01E08.nfo
-rwxrwxr-x    1 21023    30502         1058 Mar 30  2020 season.nfo
/config #

 

The Docker image always chowns /config recursively on every start up. The logs show the failure occurs after ffmpeg writes images into /config/cache/temp/<UUID>/:

ProcessRun 'quick-extract-imageseries' Process exited with code 0

 

Here are the contents of /config/cache/temp during a thumbnail image extraction from nothing in the cache directory, during image extraction, and then after the failure (once image extraction completes the directory is removed):

/~ # ls -al / | egrep -w config
drwxrwxr-x   17 21023    30502           17 Jan 31  2020 config
/~ #
/~ # ls -al /config/ | egrep -w cache
drwxrwxr-x   20 21023    30502           23 Apr 28 05:00 cache
/~ #
/~ # ls -al /config/cache/ | egrep -w temp
drwxr-xr-x    2 21023    30502            2 Apr 28 13:41 temp
/~ #
/~ # ls -al /config/cache/temp/
total 9
drwxr-xr-x    2 21023    30502            2 Apr 28 11:00 .
drwxrwxr-x   20 21023    30502           23 Apr 28 05:00 ..
/~ # ls -al /config/cache/temp/
total 10
drwxr-xr-x    3 21023    30502            3 Apr 28 13:40 .
drwxrwxr-x   20 21023    30502           23 Apr 28 05:00 ..
drwxr-xr-x    2 21023    30502            2 Apr 28 13:40 cc5ed410e0b44d72823fd80ca9922c78
/~ #
/~ # ls -al /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/
total 8
drwxr-xr-x    2 21023    30502           15 Apr 28 13:40 .
drwxr-xr-x    3 21023    30502            3 Apr 28 13:40 ..
-rw-r--r--    1 21023    30502          622 Apr 28 13:40 img_00001.jpg
-rw-r--r--    1 21023    30502        11707 Apr 28 13:40 img_00002.jpg
-rw-r--r--    1 21023    30502        18407 Apr 28 13:40 img_00003.jpg
-rw-r--r--    1 21023    30502        12906 Apr 28 13:40 img_00004.jpg
-rw-r--r--    1 21023    30502        22876 Apr 28 13:40 img_00005.jpg
<SNIPPED_OUTPUT>
-rw-r--r--    1 21023    30502         4457 Apr 28 13:41 img_00290.jpg
-rw-r--r--    1 21023    30502         8015 Apr 28 13:41 img_00291.jpg
-rw-r--r--    1 21023    30502         5368 Apr 28 13:41 img_00292.jpg
/~ #
/~ # ls -al /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/
ls: /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/: No such file or directory
/~ #

 

Link to post
Share on other sites
Q-Droid

Is media the default group for the user media and are you able to impersonate media to navigate and perform similar operations from the host OS - create, delete, modify, etc?

What is the host OS? Are all of the extraction jobs failing? What about media access and playback? Are the NFS shares conventional POSIX filesystems on LAN served from other LInux or something else?

 

 

Link to post
Share on other sites
billg
Posted (edited)

The UID maps to 'svc_media' and the GID maps to 'media_users' outside the container. Using sudo outside of the Emby container has no issues.

$ pwd
/mnt/nas/media/tv/Series.2020/S01
$
$ sudo -u svc_media sh -c 'echo "this_is_a_test" > dummy_file'; sudo -u svc_media md5sum dummy_file; sudo -u svc_media cat dummy_file; sudo -u svc_media rm dummy_file
4cab98e0196fd0c3ea93d8c64d09eb83  dummy_file
this_is_a_test
$

The Docker host is CentOS. The image extraction job fails for any media file that is currently missing a Roku BIF file. I haven't noticed any other scheduled task failing. No media playback issues; they can watch any media file where Emby fails to create a Roku BIF file via any client including a Roku. Media is stored on a TrueNAS appliance in different ZFS pools. All Docker data, including Emby's config directory, is also mounted via NFS from this appliance and is stored in its own ZFS pool. The appliance provides NFS services to various clients beyond just the Docker host.

Edited by billg
Link to post
Share on other sites
Q-Droid

I don't see anything wrong with the permissions. It looks like the emby server process UID is the one that created and owns the other metadata files in those directories while the media files and structure are owned by different UIDs, all having a common GID. 

It seems to be specific to the image extraction since other metadata is being created by the same server process. Perhaps a different file operation from the others.

This is an older thread with a problem similar to yours.

 

Link to post
Share on other sites
billg

Changing the owner of the impacted series, from the series' root directory and down into the season directory, to the same UID Emby runs under did not resolve the issue (Emby was stopped prior to the change). Moving image thumbnail creation to the global metadata folder did work. Moving it back fails again.

Link to post
Share on other sites
Q-Droid

Are the mount options for the config share the same as for the media shares?  Same export options? 

 

 

Link to post
Share on other sites
billg

Yes, mount/export options are the same. Again no other scheduled tasks have any issue writing data into the exact same media folders. All Emby libraries are configured to store supplemental (artwork, video previews, subtitles) data in the media folder in addition to episode nfo files.

Link to post
Share on other sites
Q-Droid

I suspect this is more of a .Net Core issue with specific file operations on network shares when the runtime process doesn't have Full Control (Windows style) permissions on the destination. But I don't have proof, only your info and the many threads/posts in these forums with very similar problems. Some of them resolved by specifying the UID/GID at mount time though for SMB/CIFS shares. NFS is different and you might have to try anonuid/anongid on the export to see if that makes a difference by freeing up some of the access restrictions on the client.

It's all I can think of.

 

Link to post
Share on other sites
  • 3 weeks later...
Luke

@billg yes that is what I believe as well. Once we get through the 4.6 release we'll be updating our embedded version of the .net core runtime, so we'll see if that may help with this.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...