billg 1 Posted April 27, 2021 Share Posted April 27, 2021 (edited) The Docker Emby instance I am running is struggling to create Roku thumbnails for all media libraries. All media libraries are mounted via NFS on the underlying host OS. The host OS has no issues reading/writing files on the NFS share. Running Emby in Docker it is able to create the various media metadata and the poster.jpg files and read/write/remove files in /config. The container runs as UID 21023 (media user UID outside the container) and GID 30502 (media group GID outside the container); set as environment variables to Docker (along with media paths): [root@dockerhost ~]# docker ps | grep embyserver 1797bf5e5d0d emby/embyserver:latest "/init" About an hour ago Up About an hour 1900/udp, 8096/tcp, 7359/udp, 8920/tcp emby [root@dockerhost ~]# [root@dockerhost ~]# docker exec -it emby /bin/ash / # / # cd /config /config # ps auxfww |grep Emby 194 21023 2:56 /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3 2143 root 0:00 grep Emby /config # echo $UID 21023 /config # echo $GID 30502 /config # Sample log when extracting thumbnails. Ffmpeg successfully populates the cache folder with images. 2021-04-27 12:53:02.094 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/media/tv/Series.2020/S01/Series.2020.S01E08.mkv" -an -sn -s 320x180 -vsync cfr -r 0.1 -f image2 "/config/cache/temp/2766f0b8faa944a9afab63c4bee591aa/img_%05d.jpg" 2021-04-27 12:53:02.095 Debug App: ProcessRun 'quick-extract-imageseries' Started. 2021-04-27 12:53:40.691 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 0 2021-04-27 12:53:41.586 Error ChapterImagesTask: Error creating thumbnails for /media/tv/Series.2020/S01/Series.2020.S01E08.mkv *** Error Report *** Version: 4.5.4.0 Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3 Operating system: Linux version 5.11.11-arch1-1 (linux@archlinux) (gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP PREEMPT Tue, 30 Mar 2021 14:10:17 +0000 Framework: .NET Core 3.1.8 OS/Process: x64/x64 Runtime: system/System.Private.CoreLib.dll Processor count: 8 Data path: /config Application path: /system System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path is denied. ---> System.IO.IOException: Operation not permitted --- End of inner exception stack trace --- at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter) at Interop.CheckIo(Int64 result, String path, Boolean isDirectory, Func`2 errorRewriter) at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite) at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken) at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken) at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken) Source: System.IO.FileSystem TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo]) InnerException: System.IO.IOException: Operation not permitted Source: TargetSite: 2021-04-27 12:53:41.597 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/media/tv/Series.2020/S01/Series.2020.S01E09.mkv" -an -sn -s 320x180 -vsync cfr -r 0.1 -f image2 "/config/cache/temp/c0b04b9b1abf4dd995d2daede14ee5b9/img_%05d.jpg" 2021-04-27 12:53:41.599 Debug App: ProcessRun 'quick-extract-imageseries' Started. In the container these are the permissions to the sample media path above: /config # stat /media/ File: /media/ Size: 4096 Blocks: 8 IO Block: 4096 directory Device: b8h/184d Inode: 7920 Links: 7 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2021-04-27 18:07:23.000000000 Modify: 2021-04-27 17:17:26.000000000 Change: 2021-04-27 17:17:26.000000000 /config # /config # stat /media/tv/ File: /media/tv/ Size: 122 Blocks: 129 IO Block: 131072 directory Device: 42h/66d Inode: 4 Links: 116 Access: (0775/drwxrwxr-x) Uid: (20500/ UNKNOWN) Gid: (30502/ UNKNOWN) Access: 2016-12-13 16:20:00.000000000 Modify: 2021-04-26 23:30:09.000000000 Change: 2021-04-26 23:30:09.000000000 /config # /config # stat /media/tv/Series.2020/ File: /media/tv/Series.2020/ Size: 10 Blocks: 33 IO Block: 131072 directory Device: 42h/66d Inode: 164248 Links: 3 Access: (0775/drwxrwxr-x) Uid: (21125/ UNKNOWN) Gid: (30502/ UNKNOWN) Access: 2020-03-30 00:40:24.000000000 Modify: 2020-03-30 13:51:54.000000000 Change: 2020-03-30 13:51:54.000000000 /config # /config # stat /media/tv/Series.2020/S01/ File: /media/tv/Series.2020/S01/ Size: 33 Blocks: 129 IO Block: 131072 directory Device: 42h/66d Inode: 164257 Links: 2 Access: (0775/drwxrwxr-x) Uid: (21125/ UNKNOWN) Gid: (30502/ UNKNOWN) Access: 2020-03-30 00:55:00.000000000 Modify: 2021-04-27 17:53:41.000000000 Change: 2021-04-27 17:53:41.000000000 /config # /config # stat /media/tv/Series.2020/S01/Series.2020.S01E08.mkv File: /media/tv/Series.2020/S01/Series.2020.S01E08.mkv Size: 1581685395 Blocks: 3086712 IO Block: 131072 regular file Device: 42h/66d Inode: 164254 Links: 1 Access: (0775/-rwxrwxr-x) Uid: (21125/ UNKNOWN) Gid: (30502/ UNKNOWN) Access: 2020-03-30 00:53:38.000000000 Modify: 2020-03-30 00:55:32.000000000 Change: 2020-03-30 01:09:50.000000000 /config # Edited April 27, 2021 by billg Link to comment Share on other sites More sharing options...
Luke 37064 Posted April 27, 2021 Share Posted April 27, 2021 Hi, what about permissions on the parent folder of the video? It looks like the server is unable to copy files into there from the cache folder. Link to comment Share on other sites More sharing options...
Q-Droid 642 Posted April 28, 2021 Share Posted April 28, 2021 Have you checked the access to /config/cache/temp? Link to comment Share on other sites More sharing options...
billg 1 Posted April 28, 2021 Author Share Posted April 28, 2021 Agreed it looks like a permission issue but, if so, how can the metadata refresh task write into the same folder? Are these tasks launched under different UIDs/GIDs internally? Each directory's permission mode is always 775 from the volume mount point, /media/tv, all the way to the media. The UID may change depending on which user created the media file(s) however the GID is always 30502. Contents of the series directory and permissions are: /config # stat /media/tv/Series.2020/ File: /media/tv/Series.2020/ Size: 10 Blocks: 33 IO Block: 131072 directory Device: 42h/66d Inode: 164248 Links: 3 Access: (0775/drwxrwxr-x) Uid: (21125/ UNKNOWN) Gid: (30502/ UNKNOWN) Access: 2020-03-30 00:40:24.000000000 Modify: 2020-03-30 13:51:54.000000000 Change: 2020-03-30 13:51:54.000000000 /config # ls -al /media/tv/Series.2020/ total 1867 drwxrwxr-x 3 21125 30502 10 Mar 30 2020 . drwxrwxr-x 116 20500 30502 122 Apr 26 23:30 .. drwxrwxr-x 2 21125 30502 33 Apr 28 07:06 S01 -rwxrwxr-x 1 21023 30502 91371 Mar 30 2020 banner.jpg -rwxrwxr-x 1 21023 30502 352199 Mar 30 2020 fanart.jpg -rwxrwxr-x 1 21023 30502 293101 Mar 30 2020 landscape.jpg -rwxrwxr-x 1 21023 30502 65896 Mar 30 2020 logo.png -rwxrwxr-x 1 21023 30502 591428 Mar 30 2020 poster.jpg -rwxrwxr-x 1 21023 30502 253743 Mar 30 2020 season01-poster.jpg -rwxrwxr-x 1 21023 30502 3130 Mar 30 2020 tvshow.nfo /config # /config # /config # ls -al /media/tv/Series.2020/S01/ | egrep 'season.nfo|S01E08' -rwxrwxr-x 1 21023 30502 215751 Mar 30 2020 Series.2020.S01E08-thumb.jpg -rwxrwxr-x 1 21125 30502 1581685395 Mar 30 2020 Series.2020.S01E08.mkv -rwxrwxr-x 1 21023 30502 2397 Mar 30 2020 Series.2020.S01E08.nfo -rwxrwxr-x 1 21023 30502 1058 Mar 30 2020 season.nfo /config # The Docker image always chowns /config recursively on every start up. The logs show the failure occurs after ffmpeg writes images into /config/cache/temp/<UUID>/: ProcessRun 'quick-extract-imageseries' Process exited with code 0 Here are the contents of /config/cache/temp during a thumbnail image extraction from nothing in the cache directory, during image extraction, and then after the failure (once image extraction completes the directory is removed): /~ # ls -al / | egrep -w config drwxrwxr-x 17 21023 30502 17 Jan 31 2020 config /~ # /~ # ls -al /config/ | egrep -w cache drwxrwxr-x 20 21023 30502 23 Apr 28 05:00 cache /~ # /~ # ls -al /config/cache/ | egrep -w temp drwxr-xr-x 2 21023 30502 2 Apr 28 13:41 temp /~ # /~ # ls -al /config/cache/temp/ total 9 drwxr-xr-x 2 21023 30502 2 Apr 28 11:00 . drwxrwxr-x 20 21023 30502 23 Apr 28 05:00 .. /~ # ls -al /config/cache/temp/ total 10 drwxr-xr-x 3 21023 30502 3 Apr 28 13:40 . drwxrwxr-x 20 21023 30502 23 Apr 28 05:00 .. drwxr-xr-x 2 21023 30502 2 Apr 28 13:40 cc5ed410e0b44d72823fd80ca9922c78 /~ # /~ # ls -al /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/ total 8 drwxr-xr-x 2 21023 30502 15 Apr 28 13:40 . drwxr-xr-x 3 21023 30502 3 Apr 28 13:40 .. -rw-r--r-- 1 21023 30502 622 Apr 28 13:40 img_00001.jpg -rw-r--r-- 1 21023 30502 11707 Apr 28 13:40 img_00002.jpg -rw-r--r-- 1 21023 30502 18407 Apr 28 13:40 img_00003.jpg -rw-r--r-- 1 21023 30502 12906 Apr 28 13:40 img_00004.jpg -rw-r--r-- 1 21023 30502 22876 Apr 28 13:40 img_00005.jpg <SNIPPED_OUTPUT> -rw-r--r-- 1 21023 30502 4457 Apr 28 13:41 img_00290.jpg -rw-r--r-- 1 21023 30502 8015 Apr 28 13:41 img_00291.jpg -rw-r--r-- 1 21023 30502 5368 Apr 28 13:41 img_00292.jpg /~ # /~ # ls -al /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/ ls: /config/cache/temp/cc5ed410e0b44d72823fd80ca9922c78/: No such file or directory /~ # Link to comment Share on other sites More sharing options...
Q-Droid 642 Posted April 28, 2021 Share Posted April 28, 2021 Is media the default group for the user media and are you able to impersonate media to navigate and perform similar operations from the host OS - create, delete, modify, etc? What is the host OS? Are all of the extraction jobs failing? What about media access and playback? Are the NFS shares conventional POSIX filesystems on LAN served from other LInux or something else? Link to comment Share on other sites More sharing options...
billg 1 Posted April 28, 2021 Author Share Posted April 28, 2021 (edited) The UID maps to 'svc_media' and the GID maps to 'media_users' outside the container. Using sudo outside of the Emby container has no issues. $ pwd /mnt/nas/media/tv/Series.2020/S01 $ $ sudo -u svc_media sh -c 'echo "this_is_a_test" > dummy_file'; sudo -u svc_media md5sum dummy_file; sudo -u svc_media cat dummy_file; sudo -u svc_media rm dummy_file 4cab98e0196fd0c3ea93d8c64d09eb83 dummy_file this_is_a_test $ The Docker host is CentOS. The image extraction job fails for any media file that is currently missing a Roku BIF file. I haven't noticed any other scheduled task failing. No media playback issues; they can watch any media file where Emby fails to create a Roku BIF file via any client including a Roku. Media is stored on a TrueNAS appliance in different ZFS pools. All Docker data, including Emby's config directory, is also mounted via NFS from this appliance and is stored in its own ZFS pool. The appliance provides NFS services to various clients beyond just the Docker host. Edited April 28, 2021 by billg Link to comment Share on other sites More sharing options...
Q-Droid 642 Posted April 28, 2021 Share Posted April 28, 2021 I don't see anything wrong with the permissions. It looks like the emby server process UID is the one that created and owns the other metadata files in those directories while the media files and structure are owned by different UIDs, all having a common GID. It seems to be specific to the image extraction since other metadata is being created by the same server process. Perhaps a different file operation from the others. This is an older thread with a problem similar to yours. Link to comment Share on other sites More sharing options...
billg 1 Posted April 29, 2021 Author Share Posted April 29, 2021 Changing the owner of the impacted series, from the series' root directory and down into the season directory, to the same UID Emby runs under did not resolve the issue (Emby was stopped prior to the change). Moving image thumbnail creation to the global metadata folder did work. Moving it back fails again. Link to comment Share on other sites More sharing options...
Q-Droid 642 Posted April 29, 2021 Share Posted April 29, 2021 Are the mount options for the config share the same as for the media shares? Same export options? Link to comment Share on other sites More sharing options...
billg 1 Posted April 30, 2021 Author Share Posted April 30, 2021 Yes, mount/export options are the same. Again no other scheduled tasks have any issue writing data into the exact same media folders. All Emby libraries are configured to store supplemental (artwork, video previews, subtitles) data in the media folder in addition to episode nfo files. Link to comment Share on other sites More sharing options...
Q-Droid 642 Posted April 30, 2021 Share Posted April 30, 2021 I suspect this is more of a .Net Core issue with specific file operations on network shares when the runtime process doesn't have Full Control (Windows style) permissions on the destination. But I don't have proof, only your info and the many threads/posts in these forums with very similar problems. Some of them resolved by specifying the UID/GID at mount time though for SMB/CIFS shares. NFS is different and you might have to try anonuid/anongid on the export to see if that makes a difference by freeing up some of the access restrictions on the client. It's all I can think of. Link to comment Share on other sites More sharing options...
Luke 37064 Posted May 15, 2021 Share Posted May 15, 2021 @billg yes that is what I believe as well. Once we get through the 4.6 release we'll be updating our embedded version of the .net core runtime, so we'll see if that may help with this. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now