Jump to content


Photo

4.2.0.1 Beta - Roku Thumbnail images not being created.

centos Roku linux thumbnail

Best Answer Karpana , 23 May 2019 - 07:31 AM

@Luke & @Happy2Play

I have some fortuitous news to report.
It seems I've gotten this to work, though for the life of me, I can't explain why this solution was necessary.

Yesterday, on a whim, I disabled the "save thumbnails to media folder" option on one of my libraries and forced the TV episode in question to update missing metadata.
The .bif file was successfully created in the /emby/metadata folder.

I re-enabled the option to save thumbnails to media folder (and deleted the bif file in metadata) and it again failed.

the only "real" difference between the /metadata folder area and my media folder is who owned the folder. 

Keeping in mind that my media folder is mounted from my NAS, I recognized I have some control about which owner/group (on the CentOS side) the NAS is mounted as.

I went into my /etc/fstab file, changed the uid&gid from 5000/6000 (which was working through emby 4.1.0.23) and works for all everything but .bif files and changed it to 997/994 (emby's user and group).
I unmounted the NAS from CentOS and remounted it.

I then forced emby to update missing metadata, and much to my surprise, it worked!

As indicated, I can't explain why this worked, but it has.

I have a bit more testing, specifically around the creation of other files and functionality validation.  But for the time being this is a fortuittous start.


This is what my directory structure permissions look like now (note the 5000/6000 is now showing emby/emby)
I guess with this change, assuming everything else checks out, I should be able to reduce permissions *away* from 777.

I'll send an update in the next few days to let you know how everything else is holding up.

/
	drwxr-xr-x.   3 root root     17 Jan 25 19:11 mnt
	dr-xr-xr-x.  17 root root    242 Apr 29 20:03 ..
	dr-xr-xr-x.  17 root root    242 Apr 29 20:03 .

/mnt
	drwxr-xr-x.  3 root root  17 Jan 25 19:11 .
	drwxr-xr-x.  3 root root  21 Jan 25 19:11 nas
	dr-xr-xr-x. 17 root root 242 Apr 29 20:03 ..

/mnt/nas
	drwxr-xr-x. 3 root root 17 Jan 25 19:11 ..
	drwxr-xr-x. 3 root root 21 Jan 25 19:11 .
	drwxr-xr-x. 3 root root 20 Jan 25 19:13 lithium

/mnt/nas/lithium
	drwxr-xr-x. 3 root root   21 Jan 25 19:11 ..
	drwxr-xr-x. 3 root root   20 Jan 25 19:13 .
	drwxrwxrwx. 2 emby emby 8192 May 20 14:55 videos

/mnt/nas/lithium/videos
	drwxr-xr-x. 3 root root       20 Jan 25 19:13 ..
	drwxrwxrwx. 2 emby emby        0 May 16 22:59 TV Shows
	drwxrwxrwx. 2 emby emby     8192 May 20 14:55 .

/mnt/nas/lithium/videos/TV Shows
	drwxrwxrwx. 2 emby emby 40960 May 16 22:59 .
	drwxrwxrwx. 2 emby emby  8192 May 20 14:55 ..
	drwxrwxrwx. 2 emby emby     0 Apr 23 17:55 Star Trek - Discovery

/mnt/nas/lithium/videos/TV Shows/Star Trek - Discovery
	drwxrwxrwx. 2 emby emby   4096 Apr 23 17:55 .
	drwxrwxrwx. 2 emby emby  40960 May 16 22:59 ..
	drwxrwxrwx. 2 emby emby      0 May 20 14:43 Season 02

/mnt/nas/lithium/videos/TV Shows/Star Trek - Discovery/Season 02
	drwxrwxrwx. 2 emby emby       4096 Apr 23 17:55 ..
	drwxrwxrwx. 2 emby emby      24576 May 20 14:43 .
Go to the full post


  • Please log in to reply
42 replies to this topic

#1 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 09:22 AM

Good Morning,

 

As i was watching TV yesterday on my Roku Ultra, I noticed that many of my most recent media files (last week or two) were missing their fast-forward thumbnails.  I checked my media drive and confirmed that these videos were missing their .bif files.

This morning, I upgraded the emby server to 4.2.0.1 and manually triggered the following two scheduled tasks

1) Thumbnail image extraction
2) Scan Media Library

Neither of these resulted in the .bif files being created.

I'm attaching a screen shot of my library configuration for this specific library.
All my libraries use a nearly identical configuration.

Attached File  Sshot.png   33.93KB   6 downloads
 


Edited by Karpana, 23 April 2019 - 03:07 PM.


#2 Sammy OFFLINE  

Sammy

    Advanced Member

  • Members
  • 2724 posts
  • Local time: 04:57 AM

Posted 23 April 2019 - 09:44 AM

Mine took nearly a day to generate bif files.

Sent from my SM-G960U1 using Tapatalk

#3 mickle026 OFFLINE  

mickle026

    Advanced Member

  • Members
  • 131 posts
  • Local time: 12:57 PM

Posted 23 April 2019 - 09:52 AM

You need to run the scheduled task for media that already exists, the settings from your screenshot are for newly added media.


Edited by mickle026, 23 April 2019 - 09:52 AM.

  • Happy2Play likes this

#4 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 03:06 PM

@mickle026

 

Is there something more that I need to run beyond the two items I already mentioned that I ran, that seemed to have zero effect/impact?

 

 

This morning, I upgraded the emby server to 4.2.0.1 and manually triggered the following two scheduled tasks

1) Thumbnail image extraction
2) Scan Media Library



#5 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:09 PM

I managed to get Emby to "try" to create the BIF file earlier this evening... but I had to "force it" by searching for missing metadata in the library.
Here's a snippet of the log file.

It seems to be indicating an error accessing a directory or file, but I can't ascertain which diretory or file it is that is not accessible, nor why it has suddently (?) changed in the last week or two.

Any thoughts on which directory is at issue here?

2019-04-23 17:57:46.380 Error App: Error creating thumbnails for /mnt/nas/lithium/videos/TV Shows/Star Trek - Discovery/Season 02/Star Trek- Discovery - S02E14 - Such Sweet Sorrow (2) [WEBDL-1080p].mkv
        *** Error Report ***
        Version: 4.2.0.1
        Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-rpm_{version}_x86_64.rpm
        Operating system: Unix 3.10.0.957
        64-Bit OS: True
        64-Bit Process: True
        User Interactive: True
        Runtime: file:///opt/emby-server/system/System.Private.CoreLib.dll
        Processor count: 4
        Program data path: /var/lib/emby
        Application directory: /opt/emby-server/system
        System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path is denied. ---> System.IO.IOException: Operation not permitted
           --- End of inner exception stack trace ---
           at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
           at Interop.CheckIo(Int64 result, String path, Boolean isDirectory, Func`2 errorRewriter)
           at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
           at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
           at Emby.Server.Implementations.IO.ManagedFileSystem.CopyFile(String source, String target, Boolean overwrite)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet(Video item, LibraryOptions libraryOptions, Int32 width, CancellationToken cancellationToken)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets(Video item, LibraryOptions libraryOptions, CancellationToken cancellationToken)
           at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages(Video item, LibraryOptions libraryOptions, IDirectoryService directoryService, List`1 chapters, Boolean extractImages, Boolean saveChapters, CancellationToken cancellationToken)
        Source: System.IO.FileSystem
        TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo])
        InnerException: System.IO.IOException: Operation not permitted
        Source:
        TargetSite:

of the directories listed in this snippet of log...
/var/lib/emby/ is owned by emby.emby and is fully writeable

/opt/emby-server/system/ is owned by root.root, but I don't see this having changed in any way.

/mnt/nas/lithium/videos/ is mounted as fully read/write by emby and I can see that it successfully created the "-thumb.jpg" for this file, so it evidently has write access.

I've also confirmed that the videos directory has about 1 TB free.
 


Edited by Karpana, 23 April 2019 - 10:21 PM.


#6 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 136081 posts
  • Local time: 07:57 AM

Posted 23 April 2019 - 10:21 PM

That is an error copying the bif from it's temporary location to the location that you've chosen to save them in. So in other words the server is being denied write access.



#7 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:22 PM

and an additional note, my log files only go back a few days (back to the 20th), but I see similar errors under 4.1.0.25



#8 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:23 PM

Can you help pin down the "source" directory and the "target" directory?   

I'll try to do some permissions changes to iron this out.



#9 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:24 PM

if it's in fact the "copy" or "move", how come the -thumb.jpg file is being created successfully, but the .bif file isn't?  they're the same target location?



#10 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 136081 posts
  • Local time: 07:57 AM

Posted 23 April 2019 - 10:24 PM

The path to the media file is listed at the top of the log snippet you provided.



#11 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 136081 posts
  • Local time: 07:57 AM

Posted 23 April 2019 - 10:25 PM

if it's in fact the "copy" or "move", how come the -thumb.jpg file is being created successfully, but the .bif file isn't?  they're the same target location?

 

I don't know why your system is denying write access in that instance, I can only tell you what I'm seeing, and that's what I'm seeing.


  • Sammy likes this

#12 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:29 PM

As a precaution, what's the source directory?
I'll try to validate permissions on both ends.

#13 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 10:50 PM

I've been able to confirm that the emby user is able to write files to the target directory.

I'm starting to wonder if there's something up with the source file, though I don't know where that is.

#14 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 15347 posts
  • Local time: 04:57 AM
  • LocationWashington State

Posted 23 April 2019 - 10:55 PM

Aren't the built in \cache\temp?  Do you have "quick-extract-imageseries" logs in your log folder.



#15 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 23 April 2019 - 11:00 PM

Here's the file list from /var/lib/emby/logs

[media@Helium ~]$ cd /var/lib/emby/logs
[media@Helium logs]$ ls -altr
total 18124
-rw-r--r--.  1 emby emby 1171635 Apr 20 23:59 embyserver-63691401600.txt
-rw-r--r--.  1 emby emby 1962615 Apr 22 00:00 embyserver-63691488000.txt
-rw-r--r--.  1 emby emby   38103 Apr 22 20:22 ffmpeg-remux-695667a2-fcc0-44f5-9b4e-ccd8f25
660f8_1.txt
-rw-r--r--.  1 emby emby 1278421 Apr 23 00:00 embyserver-63691574400.txt
drwxr-xr-x. 13 emby emby     169 Apr 23 00:00 ..
-rw-r--r--.  1 emby emby  353795 Apr 23 06:26 embyserver-63691597584.txt
-rw-r--r--.  1 emby emby   25631 Apr 23 06:26 embyserver-63691597591.txt
-rw-r--r--.  1 emby emby  199281 Apr 23 06:26 hardware_detection-63691597597.txt
-rw-r--r--.  1 emby emby  103860 Apr 23 06:39 embyserver-63691598382.txt
-rw-r--r--.  1 emby emby  199283 Apr 23 06:39 hardware_detection-63691598389.txt
-rw-r--r--.  1 emby emby  325934 Apr 23 08:23 embyserver-63691604633.txt
-rw-r--r--.  1 emby emby  199283 Apr 23 08:24 hardware_detection-63691604640.txt
-rw-r--r--.  1 emby emby 1697523 Apr 23 16:47 embyserver-63691634951.txt
drwxr-xr-x.  2 emby emby    4096 Apr 23 16:49 .
-rw-r--r--.  1 emby emby  199283 Apr 23 16:49 hardware_detection-63691634960.txt
-rw-r--r--.  1 emby emby 8693206 Apr 23 21:58 embyserver.txt
[media@Helium logs]$



#16 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 24 April 2019 - 09:27 AM

As I continue to try to figure this out ...

 

I've been able to confirm that the process that creates the various .jpg files in /var/lib/cache/temp/<identifier> is working, as i can see the jpg files being created.

I can also see the -thumb.jpg file being created within the same directory structure for all my videos (as noted earlier).

i guess the question I'm struggling with is this

  • Is the .bif file created in the same /var/lib/cache/temp/<identifier> folder?


as a side note, I believe this is the command line being executed when the system tries to create the .bif file.  This was pulled out from the "ps auxwwww" command.  I'm having trouble capturing the move/copy command (if any), though I'm having trouble running it myself from a command prompt (likely due to the spaces and parentheses in the filenames).

/opt/emby-server/bin/ffmpeg -f matroska -skip_interval 10 -copyts -i file:/mnt/nas/lithium/videos/TV Shows/Star Trek - Discovery/Season 02/Star Trek- Discovery - S02E14 - Such Sweet Sorrow (2) [WEBDL-1080p].mkv -an -sn -s 320x173 -vsync cfr -r 0.1 -f image2 /var/lib/emby/cache/temp/9420f8b9d18f482889c98e5802490841/img_%05d.jpg

 



#17 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 24 April 2019 - 10:02 AM

So I decided to do some debugging, and installed the inotifywait command on my emby server, and proceeded to put a recursive watch on the /var/lib/emby/cache/temp directory.

I'm attaching a complete log of the results, but in summary:
  • creates a directory in /var/lib/emby/cache/temp called 97ff624674404b72ab4b1e8c9b96385e
  • creates ~400 jpg files in this directory
  • creates a file called 126e977d40a9480ebfe645279c1cd9d5 in this directory
  • proceeds to read eah of the ~400 jpg files, and for each one, modifies the 126e977d40a9480ebfe645279c1cd9d5 file
  • closes the file called 126e977d40a9480ebfe645279c1cd9d5
  • gets a directory listing of the 97ff624674404b72ab4b1e8c9b96385e directory
  • proceeds to delete all of the jpgs files therein
  • proceeds to delete the 126e977d40a9480ebfe645279c1cd9d5 file
  • proceeds to delete the 97ff624674404b72ab4b1e8c9b96385e directory
Attached File  CacheTempDirectory_iNotify.txt   296.14KB   2 downloads
 
Admittedly, this watch of the temp directory doesn't show me where the "move" or "copy" command is, as an earlier test showed me that if a process can't do an action on a given file/directory, the inotify tools shows nothing.
 
So I decided to do this test again with a watch on the target video directory for the .bif file.  (again log attached,but only a partial one... just the last 400 lines)
Attached File  VideoTargetDirectory_iNotify.txt   46.18KB   2 downloads
 
When I reviewed this second log, there seems to be some very strange behaviour.
  • As expected, there's 1000s of "read actions" as the source video file is read.
  • This video file is then closed, and then followed by hundreds of writes to the .bif file (in the target video directory)
  • This is followed with a close command, of the .bif file, (still within the target directory.)
  • And then a final action to delete the just created .bif file (within the target video directory).
 
Why is emby deleting the .bif file it just created in the target video directory?
 
 
 
 


#18 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 25 April 2019 - 09:21 AM

Every indication is that this isn't a "write denied problem", as indicated by @Luke.  The two inotifywatch scans I provided yesterday are, in my opinion, sufficient proof that not only is there not a write access issue, but there's something else at work here.  Specifically, it seems that emby is outright deleting the .bif file that it just finished creating in the target directory.

My programming spidey senses are leading me to believe that the "Operation not permitted" message that I'm seeing isn't an indication of "write access error", but rather, maybe this is an indication that the move/copy operation is failing because of something else.  Perhaps that the .bif file can't be moved from /cache/temp into the video directory because it doesn't exist in the /cache/temp directory structure.  

 

This seems to be supported by the inotifywatch scan of the target directory (that I provided yesterday) which seems to indicate that the .bif file is being actively created within the target video directory.

 

What really confuses me, is why the created .bif file (which is already in the target video directory) is then being subsequently deleted.

 

@Happy2Play, is there anything else you can suggest I take a peek at to help debug this problem?



#19 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 26 April 2019 - 09:30 AM

I decided to go back and see how far back this issue has existed in my system, and determined that this problem started on/after the April 7th 2019 10:35am, which was the last .bif file my Emby Server created.

Looking back at my install history, since I try to update to the most recent emby beta as soon as they come out:

  • April 6th 9:28pm   ---> Upgraded from 4.1.0.20 to 4.1.0.22
  • April 9th 6:21am   ---> Upgraded from 4.1.0.22 to 4.1.0.23
  • April 12th 4:45pm ---> Upgraded from 4.1.0.23 to 4.1.0.24
  • April 17th 6:44pm ---> Upgraded from 4.1.0.24 to 4.1.0.25
  • April 23rd 6:26pm ---> Upgraded from 4.1.0.25 to 4.2.0.1

Based on these install timings, I would say that whatever issue is causing my .bif files to no longer get created, started with 4.1.0.23.

 

Please advise what steps I can do to help iron out this issue?



#20 Karpana OFFLINE  

Karpana

    Advanced Member

  • Members
  • 74 posts
  • Local time: 06:57 AM

Posted 02 May 2019 - 05:05 PM

I'm continuing to look for guidance on this issue.

@Luke has suggested this is a permissions problem. 

My investigation has determined this is likely not the case as

  • The presumable source dirctory, /cache/temp, is owned by emby and Emby is successfully able to create, manipulate and remove file from this structure
  • The presumable target directory (which is my vides structure) is writeable as
    • Emby is successfuly able to create the fanart.jpg, landscape.jpg, logo.png, poster.jpg, as well as store the various subtitle files in the same directory as the video file.
    • Further, for TV shows,Emby is also successfully able to create the <name>-thumb.jpg file in the same directory.

What additional information would need to be provided to help debug this issue?
What additional things should I be trying?

I've previously provided a fair amount of information about what's going on within the filesystem (using the linux inotify tool) 

 

Please help.







Also tagged with one or more of these keywords: centos, Roku, linux, thumbnail

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users