Error creating thumbnails / System.IO.IOException: Invalid argument (emby running in docker container)

[OKDaG 1]

Hello,

my emby docker installation is not able to create any thumbnails, i've checked every permissions twice but i don't get it working. Logs attached.

Error is:

2021-01-17 10:11:07.001 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/srv/ftp/Movies-SD/Ein Käfer auf Extratour (1973) [imdbid=tt0070280].mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2021-01-17 10:11:07.058 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0
2021-01-17 10:11:07.072 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/srv/ftp/Movies-SD/Ein Käfer auf Extratour (1973) [imdbid=tt0070280].mkv" -an -sn -s 320x192 -vsync cfr -r 0.1 -f image2 "/srv/local-persistent/emby/cache/temp/10407127a1cd43b1b8231efe100e9183/img_%05d.jpg"
2021-01-17 10:11:17.803 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 0
2021-01-17 10:11:18.209 Error App: Error creating thumbnails for /srv/ftp/Movies-SD/Ein Käfer auf Extratour (1973) [imdbid=tt0070280].mkv
    *** Error Report ***
    Version: 4.5.4.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 5.4.0-58-generic (buildd@lcy01-amd64-004) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020
    Framework: .NET Core 3.1.8
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 4
    Data path: /config
    Application path: /system
    System.IO.IOException: System.IO.IOException: Invalid argument
       at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
       at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
       at Emby.Server.Implementations.IO.ManagedFileSystem.CopyFile(String source, String target, Boolean overwrite)
       at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken)
       at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken)
       at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken)
    Source: System.IO.FileSystem
    TargetSite: Void CopyFile(System.String, System.String, Boolean)

 

# docker-compose.yml

version: '2'

services:
  emby:
    image: emby/embyserver
    container_name: emby
    restart: always
    devices:
       - /dev/dri:/dev/dri
    ports:
      - 8096:8096
    volumes:
      - /srv/ftp:/srv/ftp:rw
      - /srv/local-persistent/emby:/srv/local-persistent/emby:rw
      - /opt/config/emby:/config:rw
      - /etc/localtime:/etc/localtime:ro
    environment:
      - UID=1000
      - GID=1000
      - GIDLIST=1000,44,109
      - TZ=Europe/Berlin

embyserver.txt permissions.txt

[OKDaG 1]

this is with activated debug logging (just one file, all files have the same error, no bif is created):
 

2021-01-17 22:45:46.178 Debug XmlSerializer: Deserializing file /config/root/default/Filme-SD/options.xml
2021-01-17 22:45:46.335 Debug App: Running CollectionFolderImageProvider for /config/root/default/Filme-SD
2021-01-17 22:45:46.362 Debug App: Setting image url into item 18
2021-01-17 22:45:46.474 Debug App: Running FolderImageProvider for /srv/ftp/Movies-SD
2021-01-17 22:45:46.701 Debug XmlSerializer: Deserializing file /config/config/metadata.xml
2021-01-17 22:45:46.797 Debug App: Running FFProbeProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:45:46.863 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2021-01-17 22:45:46.891 Debug MediaProbeManager: ProcessRun 'ffprobe' Started.
2021-01-17 22:45:49.843 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0
2021-01-17 22:45:50.080 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv" -an -sn -s 320x190 -vsync cfr -r 0.1 -f image2 "/srv/local-persistent/emby/cache/temp/540c5802eea540929106765fdf4f0666/img_%05d.jpg"
2021-01-17 22:45:50.101 Debug App: ProcessRun 'quick-extract-imageseries' Started.
2021-01-17 22:46:54.213 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 0
2021-01-17 22:46:54.798 Error App: Error creating thumbnails for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
	*** Error Report ***
	Version: 4.5.4.0
	Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
	Operating system: Linux version 5.4.0-58-generic (buildd@lcy01-amd64-004) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020
	Framework: .NET Core 3.1.8
	OS/Process: x64/x64
	Runtime: system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: /config
	Application path: /system
	System.IO.IOException: System.IO.IOException: Invalid argument
	   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
	   at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.CopyFile(String source, String target, Boolean overwrite)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken)
	Source: System.IO.FileSystem
	TargetSite: Void CopyFile(System.String, System.String, Boolean)

2021-01-17 22:46:54.840 Debug App: Running MovieDbProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:54.891 Info HttpClient: GET https://api.themoviedb.org/3/movie/102712?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers&language=de&include_image_language=de,null,en
2021-01-17 22:46:55.065 Debug App: Throttling Tmdb by 124 ms
2021-01-17 22:46:55.190 Info HttpClient: GET https://api.themoviedb.org/3/movie/102712?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers&language=en&include_image_language=de,null,en
2021-01-17 22:46:55.248 Debug App: Throttling Tmdb by 241 ms
2021-01-17 22:46:55.491 Info HttpClient: GET https://api.themoviedb.org/3/configuration?api_key=f6bd687ffa63cd282b6ff2c6877f2669
2021-01-17 22:46:55.505 Info App: MovieDb settings: {"images":{"base_url":"http://image.tmdb.org/t/p/","secure_base_url":"https://image.tmdb.org/t/p/","backdrop_sizes":["w300","w780","w1280","original"],"logo_sizes":["w45","w92","w154","w185","w300","w500","original"],"poster_sizes":["w92","w154","w185","w342","w500","w780","original"],"profile_sizes":["w45","w185","h632","original"],"still_sizes":["w92","w185","w300","original"]},"change_keys":["adult","air_date","also_known_as","alternative_titles","biography","birthday","budget","cast","certifications","character_names","created_by","crew","deathday","episode","episode_number","episode_run_time","freebase_id","freebase_mid","general","genres","guest_stars","homepage","images","imdb_id","languages","name","network","origin_country","original_name","original_title","overview","parts","place_of_birth","plot_keywords","production_code","production_companies","production_countries","releases","revenue","runtime","season","season_number","season_regular","spoken_languages","status","tagline","title","translations","tvdb_id","tvrage_id","type","video","videos"]}
2021-01-17 22:46:55.560 Debug App: Running OmdbItemProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:55.583 Info HttpClient: GET https://private.omdbapi.com?apikey=fe53f97e&i=tt0072361&plot=short&tomatoes=true&r=json
2021-01-17 22:46:57.377 Debug App: Running MovieNfoProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:57.417 Debug App: MovieNfoProvider returned no metadata for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:57.436 Debug App: Running MovieDbImageProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:57.474 Info HttpClient: GET https://api.themoviedb.org/3/movie/102712?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers
2021-01-17 22:46:57.509 Info HttpClient: GET https://image.tmdb.org/t/p/original/iyYnaxnUD4KmyDgwqPLpkwROBdr.jpg
2021-01-17 22:46:57.578 Debug ProviderManager: Saving image to /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361]-poster.jpg
2021-01-17 22:46:57.616 Info HttpClient: GET https://image.tmdb.org/t/p/original/1kQRUNjsVJ5tsa6FIPDTjNiihZc.jpg
2021-01-17 22:46:57.642 Debug ProviderManager: Saving image to /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361]-fanart.jpg
2021-01-17 22:46:57.670 Debug App: Running OmdbImageProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:57.679 Debug App: Running FanartMovieImageProvider for /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv
2021-01-17 22:46:58.050 Debug ProviderManager: Saving /srv/ftp/Movies-SD/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv to Nfo.

 

[Q-Droid 989]

You have special characters in your file name. I wonder if they are being interpreted by the by the commands during the scan process.

Quote

Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv

 

[OKDaG 1]

tried it already without any brackets and whitespaces and also with privileged mode on, everytime the same error.

[OKDaG 1]

log of one example of my tests, this was also one of my first thoughts... and checked before posting here
 

2021-01-17 09:59:56.687 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0
2021-01-17 09:59:56.721 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/srv/ftp/Movies-SD/Futurama_Leela_und_die_Enzyklopoden.mkv" -an -sn -s 320x174 -vsync cfr -r 0.1 -f image2 "/srv/local-persistent/emby/cache/temp/23aac6f80dd84d90867ec2f441bb6311/img_%05d.jpg"
2021-01-17 10:00:01.630 Info App: Emby.Kodi.SyncQueue: Creating DB Repository...
2021-01-17 10:00:02.403 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 0
2021-01-17 10:00:02.656 Error App: Error creating thumbnails for /srv/ftp/Movies-SD/Futurama_Leela_und_die_Enzyklopoden.mkv
        *** Error Report ***
        Version: 4.5.4.0
        Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
        Operating system: Linux version 5.4.0-58-generic (buildd@lcy01-amd64-004) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020
        Framework: .NET Core 3.1.8
        OS/Process: x64/x64
        Runtime: system/System.Private.CoreLib.dll
        Processor count: 4
        Data path: /config
        Application path: /system
        System.IO.IOException: System.IO.IOException: Invalid argument
           at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
           at Interop.CheckIo(Int64 result, String path, Boolean isDirectory, Func`2 errorRewriter)
           at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken)
        Source: System.IO.FileSystem
        TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo])

2021-01-17 10:00:02.659 Info App: MovieDbProvider: Finding id for item: Futurama_Leela_und_die_Enzyklopoden
2021-01-17 10:00:02.659 Info HttpClient: GET https://api.themoviedb.org/3/search/movie?api_key=f6bd687ffa63cd282b6ff2c6877f2669&query=Futurama_Leela_und_die_Enzyklopoden&language=de

 

[Q-Droid 989]

Is /srv/ftp a local or remote volume? I can't see anything that shows what the specific problem might be and I don't know the details of the error message. I think the extracted chapter images would be written to /srv/local-persistent/emby/metadata, and that path appears to have the correct permissions.

If the media file itself is readable by UID 1000 then I don't know what else it might be.

 

 

[OKDaG 1]

all paths are local, /srv/ftp is a fuse mounted unionfs and writeable, it already works with the fanart files, ffmpeg is also able to write to the metadata dir. i don't understand the error message. i guess fuse is the problem which is strange because all the other meta files are created well, will test it next.

[Q-Droid 989]

I'm guessing, and it's only a guess, is that for some reason it can't copy the extracted image files from the transcoding temp path to the metadata path...if this is in fact what it's doing at this point. One of the devs or support team would have to confirm this.

Free space good on all volumes?

 

[Happy2Play 9780]

Not sure but debug logging may show more information.  I believe that is required to actually see the "quick-extract-imageseries" log also.

Dashboard-Logs

Note you have to restart Emby to disable this option, pretty sure not required for enabling.

[Luke 42077]

The .net core runtime seems to have trouble with unionfs. If you do a forum search you'll find other things that have been reported along these lines. In the near future we'll be updating from .net core 3.1 to 5.0. There's no guarantee it will make a difference but it's possible.

[OKDaG 1]

5 hours ago, Happy2Play said:

Not sure but debug logging may show more information.  I believe that is required to actually see the "quick-extract-imageseries" log also.

Dashboard-Logs

Note you have to restart Emby to disable this option, pretty sure not required for enabling.

Some posts above you can see it with debug logging enabled, unfortunately no more helpful informations in the log.

5 hours ago, Luke said:

The .net core runtime seems to have trouble with unionfs. If you do a forum search you'll find other things that have been reported along these lines. In the near future we'll be updating from .net core 3.1 to 5.0. There's no guarantee it will make a difference but it's possible.

ok, will test it today without unionfs or fuse, guess the error is gone then and hopefully later with the update.

and @Q-Droidyes, all filesystems have gigabytes of free space
thanks for the support so far.

[OKDaG 1]

ok, did a little test and disabled the option to save the option to save the bif files within the mediafolder, and they will be created well inside the cache path (not unionfs):
 

# find /srv/local-persistent/emby/ -name *.bif
/srv/local-persistent/emby/metadata/library/c8/c8e97aee33224c25ff132d1e7fc169e0/Ein Käfer auf Extratour (1973) [imdbid=tt0070280]-320-10.bif
/srv/local-persistent/emby/metadata/library/79/79abeeb1a54d4cdf5cb9dd8ebb265475/Das verrückteste Auto der Welt (1975) [imdbid=tt0072361]-320-10.bif

so there is somewhere a difference inside emby between putting other media files like fanart and nfo to the (unionfs) mediafolder which is working and the thumbnail bif files:

# ls /srv/ftp/Movies-SD/
'Das verrückteste Auto der Welt (1975) [imdbid=tt0072361]-fanart.jpg'
'Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].mkv'
'Das verrückteste Auto der Welt (1975) [imdbid=tt0072361].nfo'
'Das verrückteste Auto der Welt (1975) [imdbid=tt0072361]-poster.jpg'
'Ein Käfer auf Extratour (1973) [imdbid=tt0070280].mkv'
'Ein Käfer auf Extratour (1973) [imdbid=tt0070280].nfo'
'Ein Käfer auf Extratour (1973) [imdbid=tt0070280]-poster.jpg'

so creating is working, copying not... 

[Q-Droid 989]

Thinking about this strictly at the filesystem level,  creation and deletion of files are directory level (permissions) operations. Modifying files is a file level (permissions) operation. Now you add the unionfs layer with it's own policies on how to handle existing and new objects to the structure.

So, have you made sure that all of the underlying filesystems have ownership and permissions in sync with what you're expecting at the FUSE layer? Do you have any RO, RW policies that might be in conflict with existing objects or expected behavior?

 

[OKDaG 1]

sorry i don't have any policies on filesystem level, all permissions are set correctly from my understanding (allow_other and all with UID and GID 1000).
i've tested the following:

- connect to the emby container with the same uid/gid as emby inside is running
- create testfile within cache dir
- copy testfile from cache dir to media dir (unionfs)
- add something to the testfile in the media dir (unionfs)
- removing testfile from media dir (unionfs)

everything is working so far, would like to understand what exactly emby is doing when it fails. in this case i'm not able to reproduce the error outside of emby.

# docker ps|grep -i emby
bb584eef9f5f        emby/embyserver:latest              "/init"                  18 minutes ago      Up 18 minutes          1900/udp, 8096/tcp, 7359/udp, 8920/tcp                                                 emby_emby.1.umvs3ne96327g14ep5nan6d03
# docker exec -it -u 1000:1000 bb584eef9f5f sh
/ $ id
uid=1000 gid=1000
/ $ echo "test" >/srv/local-persistent/emby/cache/testfile
/ $ ls -al /srv/local-persistent/emby/cache/testfile
-rw-r--r--    1 1000     1000             5 Jan 18 19:00 testfile
/ $ cp /srv/local-persistent/emby/cache/testfile /srv/ftp/Movies-SD/
/ $ ls /srv/ftp/Movies-SD/testfile -al
-rw-r--r--    1 1000     1000             5 Jan 18 19:00 testfile
/ $ echo "test2" >>/srv/ftp/Movies-SD/testfile
/ $ cat /srv/ftp/Movies-SD/testfile
test
test2
/ $ rm /srv/ftp/Movies-SD/testfile
/ $ ls /srv/ftp/Movies-SD/testfile -al
ls: /srv/ftp/Movies-SD/testfile: No such file or directory
/ $ ps -ef
PID   USER     TIME  COMMAND
    1 root      0:00 s6-svscan -t0 /var/run/s6/services
   31 root      0:00 s6-supervise s6-fdholderd
  170 root      0:00 s6-supervise emby-server
  174 root      0:00 sh ./run
  196 1000      0:23 /system/EmbyServer -programdata /config -ffdetect /bin/ffd
  648 1000      0:00 sh
  738 1000      0:00 ps -ef
/ $ exit

 

[OKDaG 1]

Ok, tried it with emby running outside docker container with the same UID/GID and it worked! So problem is related to emby running as docker container.

[Luke 42077]

Thanks for following up !

[OKDaG 1]

So with the actual beta (from today) problem is "fixed", no more errors.

[Luke 42077]

We haven't done anything related to this, but thanks for the info.

[OKDaG 1]

Sorry was late yesterday and the files are created from the previous test without docker, so problem is also in the beta version.

[Luke 42077]

Thanks for the info. In the near future we'll update to .net core 5.0 and we'll see if that might help with this or not.