Real time monitoring for subfolders

[kaynemo 13]

Hi all, sorry if this has been asked - can't seem to find the answer. I am trying to setup a media library that has subfolders for episodes. As I setup the media library, I turn on realtime monitoring and I add subfolders to the list of folders within the library, but the realtime monitoring doesn't work for media that is uploaded to subfolders directly, unless I force to scan the library. Strange enough even when I add to the root folder of this library, the realtime monitoring also fails. How can I tackle this? Thank you

Edited October 25, 2022 by kaynemo

[Happy2Play 8252]

Can you go over a specific example.

How to Report a Problem

 

[kaynemo 13]

I am not sure. No matter what I do, when I add files to the subfolders, emby doesn't scan "realtime" these files and add them to the library. the folder we are watching is komitet, it has 4 subfolders - 1ser, 2ser, 3ser, 4ser. I need to add files to these folders, so emby sees these files and adds them to the library

embyserver.txt

Edited October 25, 2022 by kaynemo

[kaynemo 13]

In the server log it says this:

2022-10-25 12:31:06.889 Error LibraryMonitor: Error in Directory watcher for: /Volumes/Video Share/komitet
    *** Error Report ***
    Version: 4.7.8.0
    Command line: /Applications/EmbyServer.app/Contents/MacOS/EmbyServer.dll
    Operating system: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64
    Framework: .NET 6.0.5
    OS/Process: x64/x64
    Runtime: Applications/EmbyServer.app/Contents/MacOS/System.Private.CoreLib.dll
    Processor count: 8
    Data path: /Users/emby/.config/emby-server
    Application path: /Applications/EmbyServer.app/Contents/MacOS
    System.IO.IOException: System.IO.IOException: Too many changes at once in directory:{0}.
    Source:
    TargetSite:
    No Stack Trace Available

 

But I can't understand why it chooses this particular folder for this error

[kaynemo 13]

Moreso, after I restart the server, upon restart is scans this folder, but immediately stop watching this particular directory with the same error. I am at loss - any new folder I create and attempt to realtime watch it fails to do so... what can I do?

 

After full restart:

2022-10-25 13:21:10.986 Error LibraryMonitor: Error in Directory watcher for: /Volumes/Video Share/komitet
	*** Error Report ***
	Version: 4.7.8.0
	Command line: /Applications/EmbyServer.app/Contents/MacOS/EmbyServer.dll
	Operating system: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64
	Framework: .NET 6.0.5
	OS/Process: x64/x64
	Runtime: Applications/EmbyServer.app/Contents/MacOS/System.Private.CoreLib.dll
	Processor count: 8
	Data path: /Users/emby/.config/emby-server
	Application path: /Applications/EmbyServer.app/Contents/MacOS
	System.IO.IOException: System.IO.IOException: Too many changes at once in directory:{0}.
	Source: 
	TargetSite: 
	No Stack Trace Available
	
2022-10-25 13:21:10.987 Info LibraryMonitor: Stopping directory watching for path /Volumes/Video Share/komitet

 

 

I am at a complete loss..... what do I do? No changes to the folder BEFORE or AFTER restart have been made....

 

UPD1. I seem to have narrowed the problem. If I make a new folder, after make a new media library connected to the folder - everything is fine. But the minute I copy a file/s to the folder, system errors out on realtime watching...

 

UPD2. Single file copy works. Copying a folder errors out the realtime watch. Also creating library with empty subfolders works, but copying a file inside one of them crashes the realtime.

 

I am absolutely positive this USED to work. I recently upgraded emby and it seems that this is a bug and/or a recent problem

Edited October 25, 2022 by kaynemo

[Luke 37022]

3 hours ago, kaynemo said:

Moreso, after I restart the server, upon restart is scans this folder, but immediately stop watching this particular directory with the same error. I am at loss - any new folder I create and attempt to realtime watch it fails to do so... what can I do?

 

After full restart:

2022-10-25 13:21:10.986 Error LibraryMonitor: Error in Directory watcher for: /Volumes/Video Share/komitet
	*** Error Report ***
	Version: 4.7.8.0
	Command line: /Applications/EmbyServer.app/Contents/MacOS/EmbyServer.dll
	Operating system: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64
	Framework: .NET 6.0.5
	OS/Process: x64/x64
	Runtime: Applications/EmbyServer.app/Contents/MacOS/System.Private.CoreLib.dll
	Processor count: 8
	Data path: /Users/emby/.config/emby-server
	Application path: /Applications/EmbyServer.app/Contents/MacOS
	System.IO.IOException: System.IO.IOException: Too many changes at once in directory:{0}.
	Source: 
	TargetSite: 
	No Stack Trace Available
	
2022-10-25 13:21:10.987 Info LibraryMonitor: Stopping directory watching for path /Volumes/Video Share/komitet

 

 

I am at a complete loss..... what do I do? No changes to the folder BEFORE or AFTER restart have been made....

 

UPD1. I seem to have narrowed the problem. If I make a new folder, after make a new media library connected to the folder - everything is fine. But the minute I copy a file/s to the folder, system errors out on realtime watching...

 

UPD2. Single file copy works. Copying a folder errors out the realtime watch. Also creating library with empty subfolders works, but copying a file inside one of them crashes the realtime.

 

I am absolutely positive this USED to work. I recently upgraded emby and it seems that this is a bug and/or a recent problem

Maybe you have something constantly writing something underneath there?

[kaynemo 13]

16 minutes ago, Luke said:

Maybe you have something constantly writing something underneath there?

Hi, Luke! Thanks for joining... I am not sure how that could be. I was running this as a test for a project that I am about to start. I make a fresh folder, I make a fresh library. Nothing in them at all. If the folder contains no subfolders, any file added to the folder gets watched and added to the library. If I copy a subfolder with files, or if I setup the library watching subfolders in the root folder, the minute I add a file in either of the subfolders, the watching crashes. I don't think anything is writing at that moment. And I've used that before - I swear. I have a very confusing complicated setup with emby at the core and it was absolutely perfect the last project around.

Edited October 25, 2022 by kaynemo

[kaynemo 13]

Overnight the system was idle and still look at the logs - that one folder with subfolders continues to fail for real time watching

The only thing in the logs that I can see that is unique for that folder stack is this:

2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/1
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/2
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/3
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/4

But I don't understand what this means and where does the duplication happens... I have only one such stack in that location...

Seems that the only library that fails is the one that is set up for real time and has multiple subfolders

Any thoughts?

 

embyserver (1).txt

Edited October 26, 2022 by kaynemo

[Luke 37022]

2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/1
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/2
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/3
2022-10-26 14:33:30.449 Info App: Found duplicate path: /Volumes/Video Share/komitet/4

This just means you've setup multiple libraries in the server that use the same folder path or overlapping folder path, which is not officially supported at this time. That will almost assuredly lead to various quirks (but not this issue).

Anyway, one thing that won't solve this, but might make it less likely to occur, is to go through each of your emby libraries and disable the realtime monitor that you don't need it for, or that it's less important for.

If you want a solution, then you have to find out what's writing to your media folders. Emby Server is simply listening for changes, and a lot of them are happening.

[Luke 37022]

Also in times like this, when you have an issue that's difficult to trace down, removing third party plugins is a good idea just so that we can rule them out as potential causes.

[kaynemo 13]

18 hours ago, Luke said:

Also in times like this, when you have an issue that's difficult to trace down, removing third party plugins is a good idea just so that we can rule them out as potential causes.

Hey, I think I uninstalled all third-party plugins and yet still no go after restart. It is definitely the fact that there are subfolders in the main directory. Perhaps I haven't uninstalled ALL third-party plugins, but I am not sure which of them are... couldn't figure it out

 

 

embyserver (2).txt

Edited October 27, 2022 by kaynemo

[Luke 37022]

1 hour ago, kaynemo said:

Hey, I think I uninstalled all third-party plugins and yet still no go after restart. It is definitely the fact that there are subfolders in the main directory. Perhaps I haven't uninstalled ALL third-party plugins, but I am not sure which of them are... couldn't figure it out

 

 

embyserver (2).txt 67.47 kB · 0 downloads

Ok you have something outside of emby server that is making changes to files underneath /Volumes/Video Share/komitet

[kaynemo 13]

10 minutes ago, Luke said:

Ok you have something outside of emby server that is making changes to files underneath /Volumes/Video Share/komitet

I am absolutely buffled.... What could it be and how do I check it out. Also how could it be that a regular folder on the same filesystem doesn't appear to have this problem, but should I create a subfolder this problem arises? Can you nudge me in a direction to check what is writing what and where (no hidden files as well)...? I very much appreciate your help and attention, btw. One more thing - I have done the same setup with the previous version of Emby and it was dandy.

[Luke 37022]

The answer is specific to your environment so I'm afraid I don't know. Might want to look at what other software you've got installed.

[kaynemo 13]

2 hours ago, Luke said:

The answer is specific to your environment so I'm afraid I don't know. Might want to look at what other software you've got installed.

It is just a simple Linux file server network share that hasn't been changed or even upgraded for a year. It is mounted via automounter app to the Finder Volume on Mac that runs Emby. All other folders on that same share that are set to be realtime monitored work just fine until I create a subfolder.

 

So it is very strange when the folder with no subfolders runs smooth, but when you add the sub it stops. If I add these subfolders explicitly to the list of watched folders in the library config - i get "duplicates" error. But again, the strangest thing about this - this exact setup worked with the previous version of emby.

Edited October 27, 2022 by kaynemo

[kaynemo 13]

Ok, I can see that this is not something easily remedied, I guess... But I do need to setup the library correctly for the project. Is there a way to have Emby scan the specific library as a sheduled job, because the only thing the scheduler has is a full library scan... Thanks

[Happy2Play 8252]

12 hours ago, kaynemo said:

Ok, I can see that this is not something easily remedied, I guess... But I do need to setup the library correctly for the project. Is there a way to have Emby scan the specific library as a sheduled job, because the only thing the scheduler has is a full library scan... Thanks

Have you looked in the API?  There is a link at the bottom of the Dashboard but may have issues do to CORS changes.

But there are several examples of on the forum.

https://emby.media/community/index.php?/topic/112972-make-api-requests-for-library-scan-and-more-in-powershell-help/&do=findComment&comment=1190388

 

[kaynemo 13]

20 hours ago, Happy2Play said:

Have you looked in the API?  There is a link at the bottom of the Dashboard but may have issues do to CORS changes.

But there are several examples of on the forum.

https://emby.media/community/index.php?/topic/112972-make-api-requests-for-library-scan-and-more-in-powershell-help/&do=findComment&comment=1190388

 

Yes, I have read posts like this, but I have no idea how to use API.... I guess I need to read on that. Where would I start? What API is and how to edit the code?

[kaynemo 13]

Got it

Edited October 29, 2022 by kaynemo

[Luke 37022]

Let us know what you find. Thanks.

[kaynemo 13]

Well, for now I am going to have to make do with what I have - I will put all files inside one folder with no subfolders. Otherwise nothing works and I need something to work this way or another. I will try to figure out API, but since I have absolutely zero knowledge it will take time. This way or another, maybe someone from devs could try to create subfolders and build a library the way I described and see whether it works? Just to make sure it is a problem with my particular setup. Thanks for responses anyway.

[Luke 37022]

I don't think the issue is subfolders. It could be related to what I mentioned earlier about duplicating folders across libraries in emby library setup.

[kaynemo 13]

I am sorry, I don't quite get this. Let say you needed to have a library for a miniseries that would contain season subfolders. How would you go about organixing it? Is there a "how to" to read?

[Luke 37022]

Like got exactly you might have one library pointing to /TV and another to /TV/miniseries and that's what I mean about duplication. That's what's not officially supported 

[kaynemo 13]

Hmmm... still... Sorry, I must be dumb. Here is what I am trying to do. I create a folder on HD - call it GOT. Inside that fokder I create 3 folders: Season1, Season2, Season3. In Emby I create a new Library and call it Game Of Thrones and add the first folder (GOT). I scan the library and it show the subfolders, but the realtime feature doesn't work. What am I doing wrong?