Jump to content

Auto Organize - Expanding its functionality


aaronsomek

Recommended Posts

horstepipe
9 hours ago, chef said:

That might actually be a dashboard issue you have found. The plugin calls on the Emby dashboard identify popup modal there. 

I wonder if it happens in other pages on mobile as well?

 

doesn't look so, no. At least I'm able to identify movies in my movie libary on mobile, fine.

  • Thanks 1
Link to comment
Share on other sites

horstepipe

Seems to be random on some episodes, they are marked as „processing“ in auto organize and stay in that state. Any idea why this is happening?

Link to comment
Share on other sites

horstepipe

Here is an example:
I already had "Young Sheldon S04E14 in English, the file name is different to the new file which doesn't want to get organized.
Here is the log output when trying to organize the new file:
 

2021-10-30 17:41:17.669 Info App: Episode eligible file count 1
2021-10-30 17:41:17.669 Info App: Sorting file path/scheduler/Serien/Young Sheldon (2017) [imdbid-tt6226232]/Staffel 4/Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+EN]-WvF.strm
2021-10-30 17:41:17.730 Info Server: http/1.1 GET http://domain/emby/ScheduledTasks?IsEnabled=true&X-Emby-Client=Emby Web&X-Emby-Device-Name=Microsoft Edge Windows&X-Emby-Device-Id=86de916a-5aeb-4832-8347-d75c487fe915&X-Emby-Client-Version=4.7.0.13. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36 Edg/95.0.1020.38
2021-10-30 17:41:17.730 Info Server: http/1.1 GET http://domain/emby/Library/FileOrganization?StartIndex=0&Limit=50&X-Emby-Client=Emby Web&X-Emby-Device-Name=Microsoft Edge Windows&X-Emby-Device-Id=86de916a-5aeb-4832-8347-d75c487fe915&X-Emby-Client-Version=4.7.0.13. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36 Edg/95.0.1020.38
2021-10-30 17:41:17.732 Info Server: http/1.1 Response 200 to 87.132.250.176. Time: 2ms. http://domain/emby/ScheduledTasks?IsEnabled=true&X-Emby-Client=Emby Web&X-Emby-Device-Name=Microsoft Edge Windows&X-Emby-Device-Id=86de916a-5aeb-4832-8347-d75c487fe915&X-Emby-Client-Version=4.7.0.13
2021-10-30 17:41:17.733 Info Server: http/1.1 Response 200 to 87.132.250.176. Time: 3ms. http://domain/emby/Library/FileOrganization?StartIndex=0&Limit=50&X-Emby-Client=Emby Web&X-Emby-Device-Name=Microsoft Edge Windows&X-Emby-Device-Id=86de916a-5aeb-4832-8347-d75c487fe915&X-Emby-Client-Version=4.7.0.13
2021-10-30 17:41:18.143 Info App: Beginning Episode Organization
2021-10-30 17:41:18.143 Info App: Sorting file path/scheduler/Serien/Young Sheldon (2017) [imdbid-tt6226232]/Staffel 4/Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+EN]-WvF.strm into series /home/user/media/Serien/Young Sheldon
2021-10-30 17:41:18.143 Info App: Sorting file path/scheduler/Serien/Young Sheldon (2017) [imdbid-tt6226232]/Staffel 4/Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+EN]-WvF.strm to new path /home/user/media/Serien/Young Sheldon/Staffel 4/Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [Bluray-1080p Remux][8bit][h264][DTS-HD MA 5.1]-BTN.strm
2021-10-30 17:41:18.143 Info App: Perform Sorting
2021-10-30 17:41:18.143 Info App: Copying File

So the file stays in the watch folder and on GUI it stays at "processing".

I need the file names to be different to make difference between german and english episodes.

But something seems to go wrong here.

An additional option to replace episodes even if the file names are different would be nice (so just relying on the episode counter), but I guess that would break other things.

Best regards

Edited by horstepipe
Link to comment
Share on other sites

horstepipe

yeah if I remove the old S04E14, organize works.
But the root of the problem is still unclear, it is either:

That the original S04E14 has a different file name so there are some replacement problems

or

That the original S04E14 is not at the "default series libary folder". I setup new folders for auto organize and old shows/episodes are in a different folder. Unfortunately Auto Organize moves files to the old folder if the series is already there instead of creating a new folder at the specified location. Is that by design?

 

edit: I enabled "new series libary creation" fyi

Edited by horstepipe
Link to comment
Share on other sites

Okay.

There's a lot happening here.

First off, thank you for your support while we hit these bumps. It is appreciated.

 

Does the file, that is stuck in a processing state, ever get copied or moved into the appropriate library folder? 

Or is it stuck in the watched folder, in that state, and never moves?

 

I'm wondering if it has to do with the naming conventions used in the regex to try and sort the file, when there are two versions of it.

 

Would you mind posting the name of the file? 

Or you could PM me the name of that makes you feel more comfortable.

 

Link to comment
Share on other sites

25 minutes ago, horstepipe said:

yeah if I remove the old S04E14, organize works.
But the root of the problem is still unclear, it is either:

That the original S04E14 has a different file name so there are some replacement problems

or

That the original S04E14 is not at the "default series libary folder". I setup new folders for auto organize and old shows/episodes are in a different folder. Unfortunately Auto Organize moves files to the old folder if the series is already there instead of creating a new folder at the specified location. Is that by design?

 

edit: I enabled "new series libary creation" fyi

New series that emby has never seen before will be created in the folder you add to the configuration, however, if series already exist in the library, auto organize will put them in their respective folder.

 

Link to comment
Share on other sites

horstepipe
7 minutes ago, chef said:

Would you mind posting the name of the file?

hey

you see the file in the log snipped I posted.

name of old file:

Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [Bluray-1080p Remux][8bit][h264][DTS-HD MA 5.1]-BTN.strm

 

name of new file:

Young Sheldon (2017) - S04E14 - Mitchs Son and the Unconditional Approval of a Government Agency [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+EN]-WvF.strm
Link to comment
Share on other sites

horstepipe
3 minutes ago, chef said:

New series that emby has never seen before will be created in the folder you add to the configuration, however, if series already exist in the library, auto organize will put them in their respective folder.

 

yeah okay, I expected that. It would be nice then if episodes could be replaced although the file name is different. that would it make much easier for sonarr etc. users I think.

Link to comment
Share on other sites

I'm not sure we have implemented a structure that can handle multiple versions of the same tv show.

Currently, if the show exists already, and a new version of it is added to the watched folder, that version in the watched folder would be moved and copy over the version in the library. That being said,  you should be alerted in the activity log that the version in the watched folder already exists in the library (Existing Item), and shouldn't be automatically copied over.

Link to comment
Share on other sites

horstepipe
4 minutes ago, chef said:

I'm not sure we have implemented a structure that can handle multiple versions of the same tv show.

Currently, if the show exists already, and a new version of it is added to the watched folder, that version in the watched folder would be moved and copy over the version in the library. That being said,  you should be alerted in the activity log that the version in the watched folder already exists in the library (Existing Item), and shouldn't be automatically copied over.

 I enabled "Overwrite existing episode files"

Link to comment
Share on other sites

6 minutes ago, horstepipe said:

 I enabled "Overwrite existing episode files"

Okay, I know where to look now in the code.

Thank you.

Just to be clear, so I completely understand.

You have the episode in your library, but you have aquired a new version.

With "Overwrite existing episodes" enabled,  the new version should be moved into the library, copying over the old version.

This is not happening.

 

 

Link to comment
Share on other sites

horstepipe
10 minutes ago, chef said:

Okay, I know where to look now in the code.

Thank you.

Just to be clear, so I completely understand.

You have the episode in your library, but you have aquired a new version.

With "Overwrite existing episodes" enabled,  the new version should be moved into the library, copying over the old version.

This is not happening.

 

 

exactly.

Link to comment
Share on other sites

1 hour ago, horstepipe said:

exactly.

I almost have a fix @horstepipe.

Just need a little bit to test and make sure it's all good. Then I'll post here.

  • Thanks 1
Link to comment
Share on other sites

horstepipe
24 minutes ago, chef said:

I almost have a fix @horstepipe.

Just need a little bit to test and make sure it's all good. Then I'll post here.

Nice!

so the files will be replaced even if the name is different?

Link to comment
Share on other sites

Wait a minute! I found something!

@horstepipe When you select "Overwrite existing files", then click on the Activity log, and then go back to the TV configuration, is the toggle for 'Overwrite Existing files' turned off?

 

Try this:

1. Turn Off 'Overwrite Existing files

2. Click on the Activity log

3. Click back on the 'TV' tab, and see if the toggle is still Off.

 

There may be a bug in the options.

Link to comment
Share on other sites

horstepipe
17 hours ago, chef said:

Wait a minute! I found something!

@horstepipe When you select "Overwrite existing files", then click on the Activity log, and then go back to the TV configuration, is the toggle for 'Overwrite Existing files' turned off?

 

Try this:

1. Turn Off 'Overwrite Existing files

2. Click on the Activity log

3. Click back on the 'TV' tab, and see if the toggle is still Off.

 

There may be a bug in the options.

It worked here, did you maybe just forgot to press save at the bottom?

 

Link to comment
Share on other sites

TheUrbanXplorer
17 hours ago, chef said:

Okay, actually try this version.

 

It is working for me. 

Emby.AutoOrganize.zip 131.91 kB · 3 downloads

Hello @chef,
I have also tested this. I have the same problem as @horstepipe.
Something doesn't seem to work when replacing.....

By the way, there is not by chance the possibility to indicate the version with films?

So instead of:

%mn (%my) - %res.%ext
Total Recall (1990) - 1080p.mkv

For example:

%mn (%my) - %res [%???].%ext
Total Recall (1990) - 1080p [Ultimate Recall Edition].mkv

Link to comment
Share on other sites

horstepipe

log says

2021-10-31 17:19:43.337 Info App: Sorting file /home/user/media/teststrm/scheduler/Serien/My Name (2021) [imdbid-tt12940504]/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm into series /home/user/media/Serien - Sonarr/My Name
2021-10-31 17:19:43.339 Info App: Sorting file /home/user/media/teststrm/scheduler/Serien/My Name (2021) [imdbid-tt12940504]/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm to new path /home/user/media/Serien - Sonarr/My Name/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm
2021-10-31 17:19:43.339 Info App: Perform Sorting
2021-10-31 17:19:43.339 Info App: Auto organize adding /home/user/media/Serien - Sonarr/My Name/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm to inprogress list
2021-10-31 17:19:43.339 Info App: Copying File
2021-10-31 17:19:43.339 Warn App: Access to the path is denied.

 

But permisisons are okay. I definitely checked it by letting Emby delete the episode.

edit: In this case, old and new file names are the same. After letting Emby delete the old episode, auto organize sorts it as expected.

Edited by horstepipe
Link to comment
Share on other sites

59 minutes ago, horstepipe said:

log says

2021-10-31 17:19:43.337 Info App: Sorting file /home/user/media/teststrm/scheduler/Serien/My Name (2021) [imdbid-tt12940504]/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm into series /home/user/media/Serien - Sonarr/My Name
2021-10-31 17:19:43.339 Info App: Sorting file /home/user/media/teststrm/scheduler/Serien/My Name (2021) [imdbid-tt12940504]/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm to new path /home/user/media/Serien - Sonarr/My Name/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm
2021-10-31 17:19:43.339 Info App: Perform Sorting
2021-10-31 17:19:43.339 Info App: Auto organize adding /home/user/media/Serien - Sonarr/My Name/Staffel 1/My Name (2021) - S01E01 - Episode 1 [WEBDL-1080p][8bit][x264][EAC3 5.1][DE+KO]-wayne.strm to inprogress list
2021-10-31 17:19:43.339 Info App: Copying File
2021-10-31 17:19:43.339 Warn App: Access to the path is denied.

 

But permisisons are okay. I definitely checked it by letting Emby delete the episode.

edit: In this case, old and new file names are the same. After letting Emby delete the old episode, auto organize sorts it as expected.

Hmm.. 

So we can't copy over the file because the os won't let us.

We should try to delete the file first, then copy the new one into the folder.

That should be safe to do. As long as the file copies over afterwards, properly, we should be okay.

 

Is this only happening on Linux?

The reason I can't reproduce this is because I'm on windows and there doesn't seem to be any issues copying the file on my machine.

That is why I can't pinpoint the exact problem. I'm sorry guys.

Let's try to delete the Target folder file first, then move the source file, and see what happens.

 

I'll be right back! 

 

Link to comment
Share on other sites

horstepipe

Ok I‘m not at the pc at the moment. But just coming to my mind:

Do I have to enable delete media for admin user, normally this is disabled here 

Link to comment
Share on other sites

1 minute ago, horstepipe said:

Ok I‘m not at the pc at the moment. But just coming to my mind:

Do I have to enable delete media for admin user, normally this is disabled here 

We use the FileSystem to move, remove or copy the file. So it is happening at an OS level. If anything my first thoughts would be file permissions, however you have mentioned that that doesn't seem to be the case because you have allowed this in your OS.

Let's try and delete the Target file and see what happens.

 

I'll test it on my computer first, incase something goes wrong. 😆

 

BRB!

Link to comment
Share on other sites

TheUrbanXplorer
6 minutes ago, chef said:

Hmm.. 

So we can't copy over the file because the os won't let us.

We should try to delete the file first, then copy the new one into the folder.

That should be safe to do. As long as the file copies over afterwards, properly, we should be okay.

 

Is this only happening on Linux?

The reason I can't reproduce this is because I'm on windows and there doesn't seem to be any issues copying the file on my machine.

That is why I can't pinpoint the exact problem. I'm sorry guys.

Let's try to delete the Target folder file first, then move the source file, and see what happens.

 

I'll be right back! 

 

I use Windows, not Linux. 😊

  • Thanks 1
Link to comment
Share on other sites

Okay this was the original code:

                if (targetAlreadyExists || options.CopyOriginalFile)
                {
                    _logger.Info("Copying File");
                    
                    try
                    {
                        _fileSystem.CopyFile(result.OriginalPath, result.TargetPath, true); //<--Trying to copy the file here!!
                    }
                    catch (Exception ex)
                    {
                        if (ex.Message.Contains("disk space"))
                        {
                            
                            result.Status = FileSortingStatus.NotEnoughDiskSpace;
                            result.StatusMessage = "There is not enough disk space on the drive to move this file";
                        } 
                        else if (ex.Message.Contains("used by another process"))
                        {
                             
                            result.Status = FileSortingStatus.InUse;
                            result.StatusMessage = "The file maybe being streaming to a emby device. Please try again later.";
                           
                        }
                        _logger.Warn(ex.Message); //<---This is the exception we experiance (can not access file)
                        _organizationService.RemoveFromInprogressList(result);
                        _organizationService.SaveResult(result, cancellationToken);
                        EventHelper.FireEventIfNotNull(ItemUpdated, this, new GenericEventArgs<FileOrganizationResult>(result), _logger);
                        return;
                    }
                   
                

 

This is what we'll try now:

if (targetAlreadyExists || options.CopyOriginalFile)
                {
                    _logger.Info("Copying File");
                    try
                    {
                        _fileSystem.DeleteFile(result.TargetPath, true); //<-- Delete the file in the target folder
                    } catch (Exception ex)
                    {
                        _logger.Warn(ex.Message);
                    }
                    try
                    {
                        _fileSystem.CopyFile(result.OriginalPath, result.TargetPath, true); //<--Now try and copy over the new one.
                    }
                    catch (Exception ex)
                    {
                        if (ex.Message.Contains("disk space"))
                        {
                            
                            result.Status = FileSortingStatus.NotEnoughDiskSpace;
                            result.StatusMessage = "There is not enough disk space on the drive to move this file";
                        } 
                        else if (ex.Message.Contains("used by another process"))
                        {
                             
                            result.Status = FileSortingStatus.InUse;
                            result.StatusMessage = "The file maybe being streaming to a emby device. Please try again later.";
                           
                        }
                        _logger.Warn(ex.Message); //<--Hopefully no access exceptions here.
                        _organizationService.RemoveFromInprogressList(result);
                        _organizationService.SaveResult(result, cancellationToken);
                        EventHelper.FireEventIfNotNull(ItemUpdated, this, new GenericEventArgs<FileOrganizationResult>(result), _logger);
                        return;
                    }
                   
                

 

It works okay for, but it also worked okay for me without removing the target file first.

I hope we don't get any access denied errors again.

 

Also, right now we'll move the old target file to the recycle bin, until we know it works and we are all good. Then  we'll remove it all together.

Emby.AutoOrganize.zip

 

Edited by chef
  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...