Nested folder recursion, (Not scanning a library)

[centove12 0]

This is NOT about permissions, I installed the plugin, set the mount-points, added the libraries:
Movies
TV-Shows
Music
All are homed in the same dataset (as shown in the screenshot)

It happily, if slowly, scanned the TV-Shows and Music libraries.
It however refuses to find anything in the Movies folder I suspect it's because I have the movies sorted as such:

Movies/#/<Movie Name (YEAR)>/
Movies/A/<Movie Name (YEAR)>/
Movies/B/<Movie Name (YEAR)>/
Movies/C/<Movie Name (YEAR)>/
...
Movies/Z/<Movie Name (YEAR)>/

There are over 10k movies in there.

IF I add each folder (#, A-Z) it happily scans everything under that Folder, so do I have to add each folder separately?

Is there a limit on the number of folders it will recurse into?

[Luke 37064]

Hi there is no limit. This could still be about permissions. If there is a subfolder that the server is denied access to, then it will cause the parent folder to be empty.

[centove12 0]

It is NOT permissions, if I add them as seperate directories it scans just fine.

2021-03-03 10:39:44.599 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /usr/local/bin/ffprobe -i file:"/media/Videos/Movies/A/The Aviator (1985)/The Aviator (1985) - 1080p.mp4" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2021-03-03 10:39:45.564 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0

As you can see I added two sub folders and it was happy with that. So if it is supposed to traverse the directories it's not. I also don't see any errors when just /media/Video/Movies/ is added. Nothing logged, it just sits there.

[Luke 37064]

Did you see what I said though?

[centove12 0]

So explain what permission is set wrong where it can scan /media/Videos/Movies/A/* but can't scan /media/Videos/Movies/* It can also scan /media/Videos/TV-Shows/*  and /media/Music/* This is all from the same parent dataset. And when adding a folder, I can navigate to /media/Videos/Movies just fine so if it was a permissions problem wouldn't it show up when adding the folder? I can see all items in the add folder dialog. 
Does the server not log when it can't access a directory/file? 

In any case thanks for your assistance, I guess it was wasted $$$ and time. I'll look for another media server.

 

[centove12 0]

[emby@EmbyNAS /media/Videos]$ cd Movies/
[emby@EmbyNAS /media/Videos/Movies]$ ls
#    B    D    F    H    J    L    N    P    R    T    V    X    Z
A    C    E    G    I    K    M    O    Q    S    U    W    Y
[emby@EmbyNAS /media/Videos/Movies]$ whoami
emby
[emby@EmbyNAS /media/Videos/Movies]$ 
[emby@EmbyNAS /media/Videos/Movies]$ ffprobe -hide_banner -i A/Avatar\ \(2009\)/Avatar\ \(2009\)\ -\ 720p.mp4 
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'A/Avatar (2009)/Avatar (2009) - 720p.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
[...]

The emby user can read those directories just fine.

[centove12 0]

Nothing in here about it even considering anything in /media/Videos/Movies or that it can't read it or anything, nothing.

embyserver.txt

[Happy2Play 8281]

What is the content in /Movies as there could be something blocking that entire library if applying subfolders worked?

What subfolders did you add above the worked?

Edited March 4, 2021 by Happy2Play

[MRobi 159]

@centove12What do your ACL entries look like and the owner/group of those nested folders?

I look at it this way..... if it were a bug, there would have been thousands affected. Essentially everyone with nested folders for their movies. And considering the latest stable released in December, somebody would have posted about it by now. So this is likely something related to your specific setup, and that something is very likely permissions related.

[DaaNMaGeDDoN 2]

On 3/3/2021 at 4:58 PM, Luke said:

Did you see what I said though?

I did! (and why do i find this now??) And to be honest i  think i am struggling with the same or at least similar issue, maybe its a codepage thing or something, let me explain and maybe we can find the root cause of my issue, here is a recap;

Installed emby-server-deb_4.5.4.0_amd64.deb on linux Debian 10.8

Didnt want emby's home folder in /var/lib/emby so, stopped emby (systemctl stop emby blabla), moved the folder to where i wanted it (permissions unchanged), symlinked /var/lib/emby to the new location, restarted emby, emby seems to be happy. (just including all these details to be complete, dont suspect it to be related).

Ok so i did the initial setup through the webbrowser when it was time to add libs. Here is my problem; i have two huge folders /btrfs/media/series and /btrfs/media/movies one is scanned perfectly (series) the other is not. However if i change the path of the movies library to one of the movie subbfolders e.g /btrfs/media/movies/a movie, that works and for any movie in that movies folder. Also works for collections like tens of movies in a James Bond Collection folder (which in turn are all in their own separate folder under /btrfs/media/movies/James Bond Collection. Since /btrfs/media/movies is actually a symlink to /btrfs/media/-=MOVIES=-, a legacy from ntfs period which i didnt like anymore, i tried to add that folder instead, no cigar. btw i can browse to that folder in the webinterface just fine, and see all the subfolder with the collections and individual movies. The series folder is actually the same story and works fine on the first attempt via the symlink like mentioned here (/btrfs/media/series is actually a symlink to /btrfs/media/-=SERIES=-). Anyway so i figured it must be some kind of permissions issue (no reason to believe the series and movies folders are different, but hell it must be!) so i checked. Same perms. I tried removing and adding, accessing the folder via an alternative path (/btrfs/media is mounted on /mnt/media for legacy purposes), no luck. Gave the emby user a shell (usermod -s /bin/bash emby), su to emby, was able to browse the movies folder just fine. That was all a couple of days ago, i gave up.

So today i thought'd try again.

dpkg -r embyblabla

dpkg -P (purge) embyblabla.

Started over. Same issue. Series added fine. Movies are not. Manually triggering a scan seems to be occupied a while with series and doesnt seem to bother with movies.

So again i started googling on my subject, searched the forum and found this thread where you mention this particular behavior (which is a big surprise) so it MUST be a permissions issue still! (at least thats what i told myself 30 mins ago), so: i did a chmod "chmod u=rwX,g=rwX,o=rX /btrfs/media/movies/ -R" and "chown daanmageddon.media /btrfs/media/movies/ -R" (yes emby is not a member of "media", on the earlier attempt i made the mistake of adding emby to the media group and it had messed up all my 60K+ .nfo files and mediacompanion wasnt happy, thank god for btrfs and snapper). So i am now at the point where i am convinced emby can access and read all those movie files and folders just fine, just like with the series folder, however it doesnt seem to work. 

I have a hunch. On the earlier attempt i found some IOexception errors on some folders. I dont have the exact error anymore since i did a dpkg purge so no logs from the earlier attempt...actually thats not true, i looked at my google history and recall i was searching for "emby System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException"

Now, lets see if that same issue applies again, systemctl restart embyblabla, embyserver.txt is rotated and empty. Now i try to add the same movies folder, cant find any such error in the log, even after (manually) triggering an update of the movies lib. Actually there isnt any error in the log at all and emby reports its watching (if i enable real time montoring).....but no scan seems to take place, no errors either, lib is still empty.

I am quite lost. The only hunch i had was that IOexecption error which was something in the lines of "cant find part of path", but with the new attempt those errors seem to be absent anyway.

Before you start asking the obvious: no i am not adding a subfolder to the same or different library, yes of course the type of the library is movies. In the meantime i will try to add the /btrfs/media/movies folder as a series, see if that raises anything. 

However i am quite lost. Cant imagine this to be a permissions issue, but possibly some codepage issue, i remember the IOexecption errors all were about folders with special signs in their name, like a dash (-) an ampesand (&) or whatever, ill keep an eye out for them and share here if i find them. 

I really like its potential, and everything works for the series which is a start but you can imagine i really want this fixed, cant stand myself for messing up something so sorry until i am proven wrong i have to blame emby (haha sorry bad joke).

 

 

[DaaNMaGeDDoN 2]

Update: if i add the same folder as a "series" folder in emby, all the folders show up....fml

[Luke 37064]

Hi, glad to hear you're all set now. Thanks for following up.