Jump to content

Auto Organisation Bug


Vertax
 Share

Recommended Posts

Vertax
Hey guys, i think i found a Problem with the Auto Organisation.

Emby looks my Download Folder for new files to Auto Organize them, but there is a problem with automatic Archive extraction.

 

Some Media Files are in packed Archives which will automatic extract. 

Emby starts to copy the File before its completely unzipped. 

The result is that not the complete file will copied and Emby remove the Folder while the extraction is still in Progress.

 

I think a simple solution would be to check if the file size is changing in a period of time of 2 sec before the copy process starts.

 

A second thing is that the Auto Organize feature will not ignore hidden Folders and the .ignore file doesn´t work also here.
Edited by Vertax
Link to comment
Share on other sites

Hi there @@Vertax, welcome! Just so you're aware, with the next release of Emby Server the auto-organize feature will now be moved to a plugin. You can learn more about it here:

 

https://emby.media/community/index.php?/topic/49431-32265-auto-organize/

 

This will allow community members to easily jump in and contribute improvements and new features.

 

I can tell you for the first one, just be aware, there is no guaranteed way to determine if a file is currently being written to. There are things we can try but it doesn't guarantee the result will be accurate.

 

For the second thing, as far as hidden folders and .ignore files,  we never made any claims about this with auto-organize, so this isn't technically a bug.  But it might be a useful option. Thanks.

Link to comment
Share on other sites

Vertax

Thanks for the replay, it was the first thing i get in my mind to check if a file is written.

Since Emby is based on .Net there are more better ways to check this. 

I will fork the Project an play a little around.

 

Is there any Informations for Developers i could read in.

I think there are a few restrictions, because emby use mono which is only a subset of the whole .Net Framework 

Edited by Vertax
Link to comment
Share on other sites

jordy

Perhaps Emby could check the "Modified Date/Time" of each file on a schedule - every 5 or 10 minutes or so - and ensure that it is at least 2-3 (or more) minutes older that the current system time before moving/renaming it. This would give archived files time to extract completely before being organized. I use a program called Drop-it to organize files on another PC and this method (built in to Drop-It) works a treat.

Edited by jordy
Link to comment
Share on other sites

Vertax

Perhaps Emby could check the "Modified Date/Time" of each file on a schedule - every 5 or 10 minutes or so - and ensure that it is at least 2-3 (or more) minutes older that the current system time before moving/renaming it. This would give archived files time to extract completely before being organized. I use a program called Drop-it to organize files on another PC and this method (built in to Drop-It) works a treat.

This wouldn't work, the Modified Date Time is set on the start of the writing process and will not change.

The .Net Framwork has nice features to detect this, but i don´t know if mono still has implement this features.

 

I think this is a good hint to start with:

https://www.intertech.com/Blog/avoiding-file-concurrency-using-system-io-filesystemwatcher/

Edited by Vertax
Link to comment
Share on other sites

jordy

the modified date/time changes everytime a piece of data is added, removed or there is any change to the file, therefore it will - and does work. That's why it's called the "Modified" date/time...

Link to comment
Share on other sites

the modified date/time changes everytime a piece of data is added, removed or there is any change to the file, therefore it will - and does work. That's why it's called the "Modified" date/time...

 

Do your homework first, because if you did you'd see that sometimes the date modified time stamp is initially set to what the final value will be, and does not change during the transfer. All of these things vary depending on whether the transfer is local or not, or going through the network, or based on OS, file system, etc.

Link to comment
Share on other sites

jordy

I can only say that it does work, at least in my experience. I use it to organise Internet downloads - Drop-It will not process the incrementally growing download file until the modified date has stopped changing. Maybe unzipping a file is different...

Link to comment
Share on other sites

I can only say that it does work, at least in my experience. I use it to organise Internet downloads - Drop-It will not process the incrementally growing download file until the modified date has stopped changing. Maybe unzipping a file is different...

 

I'm pretty sure you're on windows and i just tested and saw behavior in windows 10 that contradicts what you're saying. copy a file from one drive to another, and you see the date modified timestamp is kept the same as the initial file you're copying from. the date created timestamp is set to the time the copy began.

Link to comment
Share on other sites

jordy

Copying a file between drives/folders etc is different as the OS already knows what the final size will be. Try it when downloading a large torrent file. Either way, all you have to do is apply the same test to any date/time field on the file, assuming of course that you are not on a snails pace machine, any size archive file should be extracted within a couple of minutes so a 5 minute delay on processing should work fine.

Link to comment
Share on other sites

chef

I just experienced this auto organize problem from the OP for the first time this morning.

 

I had never ha a issue with it in the past.

 

I had just figured that if the file was in use, emby wouldn't be able to move and rename it until the resource had finished with it.

 

Perhaps that is the answer here, don't copy the file, try to cut and paste it (move it), emby won't be able to until any other resource has released it.

 

If the organization task of emby is going to be a plugin, then it should unpackage items from one folder and move them into the organization folder.

 

From there it needs to move it into the library.

 

I'm sure a step could be skipped there.

 

I have written an app which will monitor a folder for new items, then it looks to see if the items are compressed.

 

If the items are compressed it decompresses then and moves them to auto organize monitored folder.

 

If the folder has an item which is not compressed then it moves the file to the auto organize folder.

 

Each time a file is moved or decompressed the folder it is tagged with a file called "####emby.extracted####".

 

 

Next time my app checks for new items, it ignores folders with this tag.

 

My idea is to move a file into auto organize folder and changes its name to "#{media item name}".

 

Then once the file is finished moving, remove the hashtag.

 

Have emby auto organize ignore names with hashtags.

Edited by chef
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...