Media auto detection too quick ?:)

[DingoPD 5]

The setup:

 

media disk array (xfs): all media are stored there.

media partition (zfs): only symlinks to media and metadata are stored there.

 

Only media administrator has write access to the media array, only emby and media administrator have write access to media partition, everyone else is read only.

 

when i add a new media, i simply dump the file into appropriate directory on media array, then run script that automatically creates folder structure and symbolic link for the new media on media partition, then another (secure) script sets proper permissions on the symlink and the folder structure so it can be accessed and written by the appropriate users/group.

All emby metadata is stored with the symlink on the small (50Gb) media partition (4kb clusters), instead of small files polluting the disk array with cluster size of 128kb.

 

The problem:

 

as soonest the symlink is created, emby jumps on the chance to do things with it, this happens before the secure script has a chance to change permissions, so the operation fails:

 

Script log output (sanitized):

[2018-08-06 - 17:34:45] movie_links [0.25α]: Running as user 'xxxx'
[2018-08-06 - 17:34:45] movie_links [0.25α]: Skip Link Path: /mounts/movies/xxxxxx/
[2018-08-06 - 17:34:45] movie_links [0.25α]: Skip Link Path: /mounts/movies/xxxxxx/
[2018-08-06 - 17:34:45] movie_links [0.25α]: Skip Link Path: /mounts/movies/xxxxxx/
[2018-08-06 - 17:34:45] movie_links [0.25α]: Make Link Path: /mounts/movies/xxxxxx/ --> /mounts/[movie_list]/xxxxxx/
[2018-08-06 - 17:34:45] movie_links [0.25α]: Creating movie directory: xxxxxxxxxxxx (xxxx)
Running media_perms
[2018-08-06 - 17:34:45] media_perms [0.60α]: Running as user 'xxxxxx'
[2018-08-06 - 17:34:45] media_perms [0.60α]: Working on TV Shows: XX XX XX ... done
[2018-08-06 - 17:34:47] media_perms [0.60α]: Working on Movies: XX XX XX ... done
[2018-08-06 - 17:34:47] media_perms [0.60α]: Working on Movie Link Lists: XX XX XX ... done
[2018-08-06 - 17:34:47] media_perms [0.60α]: Working on Audio Books ... done
[2018-08-06 - 17:34:47] media_perms [0.60α]: Working on Music ... done
[2018-08-06 - 17:34:47] media_perms [0.60α]: Execution time: 2.245s 

Emby log output (sanitized):

2018-08-06 17:34:45.407 Error LibraryMonitor: Error in Directory watcher for: /mounts/[movie_list]/xxxxxx
  *** Error Report ***
  Version: 3.5.2.0
  Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
  Operating system: Unix 4.9.0.4
  64-Bit OS: True
  64-Bit Process: True
  User Interactive: True
  Processor count: 8
  Program data path: /var/lib/emby
  Application directory: /opt/emby-server/system
  System.UnauthorizedAccessException: Access to the path '/mounts/[movie_list]/xxxxxx/xxxxxxxxxxxx (xxxx)' is denied. ---> System.IO.IOException: Permission denied
     --- End of inner exception stack trace ---
  System.UnauthorizedAccessException
  No Stack Trace Available
  InnerException: System.IO.IOException
  System.IO.IOException: Permission denied
2018-08-06 17:34:45.407 Info LibraryMonitor: Stopping directory watching for path /mounts/[movie_list]/xxxxxx

As you can see, it takes couple of seconds before the folder has proper permissions so that emby can use it. While emby picks up on it in less than a second (under 400ms anyway) .. which is good, but i wish i had a way to slow it down just a bit.

 

I've been trying to figure something out, but so far the solution eludes me. Media administrator script can only create new folders and symlinks, it has no access to preexisting files and folders. The permissions changing script allows no arguments for security reasons, it can find and deal with new file in media storage of ~330k files under 3 seconds, it is not fast enough for emby , and i don't think i can make it any faster than that without compromising security.

 

Is there a way to slow emby down few seconds, so the scripts have chance to do their thing please ?

 

 

 

[Luke 37113]

That's actually coming from the. Net core file system watcher. We can't stop that error but we can probably handle it better than we currently are.

[DingoPD 5]

Thanks, for the swift reply.

 

Would it be possible to grab the path from the error data, wait a bit, and then try again?

 

Also I'm fairly certain that once the error pops up, emby stops watching that particular library until the next scheduled library scan finishes or emby restarts.

 

I don't have as much time to play with it during the week, as i do over the weekends, today i started DVD rip this morning before i went to work, and then another one when i got back home, forgetting about the first one, when the second one finished, i dumped both files in the media array at once, after the scripts to set up the symlinks ran, i noticed that the error was thrown on the first file that copied, but not on the second, neither file showed in emby after about 1/2 hour. So it seems emby stops watching that library. I know, sample size of 1 doesn't make a statistic, I'll try again tomorrow night when i have couple more.

[Luke 37113]

Please try again with the next release of Emby Server, thanks.

[DingoPD 5]

will do, thanks

[DingoPD 5]

Sorry it took so long to get back to you, i've just realized to how get to the beta branch

 

2018-09-07 12:29:59.926 Info LibraryMonitor: xxxxxxxxx (/mounts/tvshows/xxxxxxxxxx) will be refreshed.

 

and after a bit it automatically starts the scan, it's pretty much perfect, thanks!

[Luke 37113]

Thanks for the feedback.