Auto Organize - Expanding its functionality

[chef 3746]

version 1.6.1.5

• Fix - Monitored folder options not saving/updating properly
• Fix - Year in identify page showed undefined if it wasn't available for the layout

DOWNLOAD: Emby.AutoOrganize_v1.6.1.5.zip

 

 

TODO: There is still an issue with Emby's ProviderManager when attempting to do a RemoteSearch on an item.

There has to be some more investigation as to why data is not returned.

 

In the follow log snippet we can see things seemingly work from inside the plugin:

 

Auto Organize successfully monitors more then one location:

2022-04-03 16:15:57.475 Info App: Auto Organize Watched Locations: 2 folder(s)

 

Auto Organize successfully locates a new file to sort:
2022-04-03 16:15:57.537 Info App: Eligible file count 1

 

Auto Organize successfully requests metadata from Providers using it's built in  ProviderManager:

First from TVDB:
2022-04-03 16:15:58.442 Info HttpClient: GET https://www.thetvdb.com/api/GetSeries.php?seriesname=Selling.the.Big.Easy&language=en
2022-04-03 16:15:58.547 Info HttpClient: GET https://www.thetvdb.com/api/GetSeries.php?seriesname=Selling+the+Big+Easy&language=en
 

Then  from MovieDb:

2022-04-03 16:15:58.620 Info HttpClient: GET https://api.themoviedb.org/3/configuration?api_key=f6bd687ffa63cd282b6ff2c6877f2669

2022-04-03 16:15:58.699 Info App: MovieDb settings: {"images":{"base_url":"http://image.tmdb.org/t/p/","secure_base_url":"https://image.tmdb.org/t/p/","backdrop_sizes":["w300","w780","w1280","original"],"logo_sizes":["w45","w92","w154","w185","w300","w500","original"],"poster_sizes":["w92","w154","w185","w342","w500","w780","original"],"profile_sizes":["w45","w185","h632","original"],"still_sizes":["w92","w185","w300","original"]},"change_keys":["adult","air_date","also_known_as","alternative_titles","biography","birthday","budget","cast","certifications","character_names","created_by","crew","deathday","episode","episode_number","episode_run_time","freebase_id","freebase_mid","general","genres","guest_stars","homepage","images","imdb_id","languages","name","network","origin_country","original_name","original_title","overview","parts","place_of_birth","plot_keywords","production_code","production_companies","production_countries","releases","revenue","runtime","season","season_number","season_regular","spoken_languages","status","tagline","title","translations","tvdb_id","tvrage_id","type","video","videos"]}
2022-04-03 16:15:58.707 Info App: MovieDbProvider: Finding id for item: Selling.the.Big.Easy
2022-04-03 16:15:58.912 Info HttpClient: GET https://api.themoviedb.org/3/search/tv?api_key=f6bd687ffa63cd282b6ff2c6877f2669&query=Selling.the.Big.Easy&language=en
2022-04-03 16:15:59.208 Info HttpClient: GET https://api.themoviedb.org/3/search/tv?api_key=f6bd687ffa63cd282b6ff2c6877f2669&query=Selling+the+Big+Easy&language=en
2022-04-03 16:15:59.265 Info HttpClient: GET https://private.omdbapi.com?apikey=fe53f97e&plot=full&r=json&s=Selling.the.Big.Easy&type=series
2022-04-03 16:15:59.604 Info HttpClient: GET https://private.omdbapi.com?apikey=fe53f97e&plot=full&r=json&i=&Episode=10&Season=2

 

However neither Providers return data, and the plugin fails to sort the new item:

2022-04-03 16:15:59.782 Warn App: No provider metadata found for Selling the Big Easy Season 2 Episode 10
2022-04-03 16:15:59.788 Error App: Error organizing file
    *** Error Report ***
    Version: 4.6.7.0
    Command line: C:\Users\MediaServer\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
    Operating system: Microsoft Windows 6.1.7601 Service Pack 1
    Framework: .NET Core 3.1.21
    OS/Process: x64/x64
    Runtime: C:/Users/MediaServer/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: C:\Users\MediaServer\AppData\Roaming\Emby-Server\programdata
    Application path: C:\Users\MediaServer\AppData\Roaming\Emby-Server\system
    Emby.AutoOrganize.Core.OrganizationException: Emby.AutoOrganize.Core.OrganizationException: No provider metadata found for Selling the Big Easy Season 2 Episode 10
      ... .... ....

 

This leads me to believe that there is something else happening in the ProviderManager, and it could be broken.

No series get returned data from the API.

[chef 3746]

 Darn! I just spent the entire day trying to debug this, and I finally figured it out.

I had updated the server.core dependency, and I probably shouldn't have done that.

 

[ttgapers 94]

7 hours ago, chef said:

version 1.6.1.5

• Fix - Monitored folder options not saving/updating properly
• Fix - Year in identify page showed undefined if it wasn't available for the layout

Just updated, cleared cache etc. - saving works well as per above. However, TV Shows at least for for manual moves show a list like:

@chef, should I delete the DB?

 

Edited April 4, 2022 by ttgapers

[chef 3746]

Just now, ttgapers said:

Just updated, cleared cache etc. - saving works well as per above. However, TV Shows at least for for manual moves show a list like:

@chef, should I delete the DB?

 

Yes, first thing tomorrow, I have a fix for all of this. It took the entire day, but I got it sorted.

[ttgapers 94]

Just now, chef said:

Yes, first thing tomorrow, I have a fix for all of this. It took the entire day, but I got it sorted.

No worries at all. Get some rest

[chef 3746]

Spent the entire day yesterday fixing this. 

DOWNLOAD: Emby.AutoOrganize_v1.6.1.6.zip

Edited April 4, 2022 by chef

[slevin7 54]

On 4/1/2022 at 6:38 PM, slevin7 said:

Thanks for your work.
But I have a small problem. if I want to assign a movie or a series, I have to assign once the whole thing again, because otherwise somehow nothing works.
But take a look 

 

 

@chefplease don't be mad. 
I found out why the selection window is broken.
It's Safari. on Chrome? no problem. Firefox? Never worked better.
Safari? well... 

Edited April 4, 2022 by slevin7

[chef 3746]

11 minutes ago, slevin7 said:

@chefplease don't be mad. 
I found out why the selection window is broken.
It's Safari. on Chrome? no problem. Firefox? Never worked better.
Safari? well... 

Is there anything in the safari dev console that would show what the issue is.

I may have to write a shim or something.

It is fixable.

[chef 3746]

On 4/2/2022 at 9:21 PM, vaise said:

I suspect I am too used to the basic one built into emby.  I had a stroke 6 months ago, and this may not be helping the situation either......

This is the first actual time i am using the enhanced version with some actual TV shows.  note, I dont use it for movies, and only TV shows I have to manually download.

I installed the new dll, restarted emby and flushed cache.

I just added 4 shows to my embyautoorganize folder.

Go to auto organize, I can see the last 4 shows.  No start/go/run button or anywhere to start the process.

In settings, I found a scheduled task option (mine set off), I had to enable the scheduled task (I always ran it manually so dont want this though).

Then I have the 'Task last run' bit in the top right corner, and a clickable refresh thing ? - I click it, it spins, but nothig on the screen.

Seems I have to refresh the browser windows to show the actual actions. 

This seems a step back - in the past, I could just manually hit go, and they would stream up the page.

In summary : On the new enhanced version, I don't seem to have the same functionality as the default one built into emby - I believe there should be a manual go button if you don't tick the scheduling option (if its there, I cant find it), and I don't see them start to process and scroll up like they used to once you click the start 'thing' at the top to start the scheduled job.  As it stands, my two issues are still there (episode naming when using 'S01E01' and picking incorrect library for TV shows if they exist in multiple locations).

 

 

 

 

 

 

 

hey @vaise

I'm working on the multiple location thing right now. I think we're going to have to add some extra configuration for when a series lives in two spots.

In order handle this properly, we can add this to sorting:

1. Attempt sorting
2. The episode in question is part of a series that has two locations
3. Don't initially sort the file.
4. Mark the file as needing user input.
5. Allow the user to use the "identify" dialog
6. The identify dialogue will have most of the information filled in already (we know what the item is), we just aren't 100% sure where to put it.
7. The identify dialogue location input will be empty.
8. Selecting the drive location, and checking the "Remember this correction" will create a smart list entry for the episode.

The next time an episode which has the similar naming structure is found, it will be place in the proper location.

The naming comparison will be based on more then just the name of the file. It will also take other naming flags in the file name into consideration.

 

Next,

I agree, the organize button should be there when the task is turned off. Let me see if I can make that happen right now.

 

[slevin7 54]

41 minutes ago, chef said:

Is there anything in the safari dev console that would show what the issue is.

I may have to write a shim or something.

It is fixable.

This is the only thing I could find

[chef 3746]

1 hour ago, slevin7 said:

This is the only thing I could find

okay, on it!

[chef 3746]

After reading vaises comment, it occurred to me that turning the task on and off in the settings page is rather redundant.

 

If you disable the task, we can't show the "Task Run" button on the Activity page...  because the task is disabled. 

 

The sorting code is tightly coupled to the Scheduled task.... like... it is the Scheduled task (for now...) 

 

So, it makes more sense to control and customize scheduled tasks in the Scheduled Tasks tab on the server. 

 

I removed the On/OFF switch from the plugin settings.

This means that the task is always on (like most other plugins, and  scheduled tasks).

 

This allows access to the task button on the Activity page, whenever.

 

Stop a Scheduled Task from running on a schedule :

You can obviously customize the scheduled task under "Scheduled Tasks" in the dashboard.

To stop the task from running on a schedule, 

Click on the scheduled task:

 

Then remove any scheduled task triggers:

 

The task will not run on a schedule, and you can then run it on command from the Activity Page. 

 

Over all, it was a cool programming trick to enable/disable the task from the configuration, but unnecessary. 

 

Let me know if I'm incorrect.

 

Edited April 4, 2022 by chef

[GOTCHA_SA 7]

Hi @chef

not sure if i am doing anything wrong but i cannot seem to get the "Overwrite existing library files" working". if i set it to on and hit save it will just revert back to off

 

[ttgapers 94]

8 minutes ago, Ryan_vl said:

not sure if i am doing anything wrong but i cannot seem to get the "Overwrite existing library files" working". if i set it to on and hit save it will just revert back to off

Also with Overwrite existing set to OFF -- in previous versions I was able to manually overwrite from admin dashboard. Has that been removed @chef, or just broken perhaps?

[chef 3746]

11 minutes ago, ttgapers said:

Also with Overwrite existing set to OFF -- in previous versions I was able to manually overwrite from admin dashboard. Has that been removed @chef, or just broken perhaps?

Okay, Ill check.

When. I moved everything into one configuration, if messed a bunch of stuff up.

[GiGo 103]

I'm still not getting things automatically moved after ticking 'remember this correction'.

[ttgapers 94]

29 minutes ago, chef said:

When. I moved everything into one configuration, if messed a bunch of stuff up.

No stress, look forward to the update

Just here helping fine tune this puppy!

[chef 3746]

2 hours ago, GiGo said:

I'm still not getting things automatically moved after ticking 'remember this correction'.

Excellent! thank you.

I made a global value, and didn't change the UI to reflect that.

I have fixed it now.

[chef 3746]

 

I've done some more research into how series are handled in the API,  when they are saved in multiple locations.

When there is more then one location, the API will return an entry for both. Giving a TotalRecordCount: 2.

Side Note: Interestingly enough, each location also gets it's own InternalId. That's good to know for another time. 

Example

{
  "Items": [
    {
      "Name": "Marvel's Daredevil",
      "ServerId": "########################",
      "Id": "120453",
      "Path": "D:\\TV\\Marvel's Daredevil (2015)",
      "RunTimeTicks": 33000000000,
      "IsFolder": true,
      "Type": "Series",
      "AirDays": [],
      "ImageTags": {
        "Primary": "cb788be2bb50cb31ac70ff9cc07c94ce",
        "Logo": "7796e70b20d1857743e1b3a7930fdbcc",
        "Thumb": "d6e69eabb6070776d3a9ff9e305ed1d7"
      },
      "BackdropImageTags": [
        "4978731797448afac5b2a24f8ef5964f",
        "17fd160959fa1e5149e1b19fc38a5089",
        "136f24e54973db9fae9c53bbaa632364",
        "aa47d64a34b499a202d4b987f265b32d",
        "879cb158bb99c8aa0e3817b1a8f27dab"
      ]
    },
    {
      "Name": "Marvel's Daredevil",
      "ServerId": "###############################",
      "Id": "179795",
      "Path": "R:\\TV\\Marvel's Daredevil (2015)",
      "RunTimeTicks": 33000000000,
      "IsFolder": true,
      "Type": "Series",
      "AirDays": [],
      "ImageTags": {
        "Primary": "0b703a1934fd606ccd2619fc96ed6461"
      },
      "BackdropImageTags": [
        "73d7721b7066e365e0dfa403bfaed406",
        "7c5138d0cdb4c02e7e99d3eed6d5ff93",
        "daf929272ebcba13d32ad5f4cc33bdfc"
      ]
    }
  ],
  "TotalRecordCount": 2
}

 

This is how we are going to decide to read the Smart List for sorting or not.

If the Series responds with an Item list larger then 1, we'll know that it has two locations, and we'll check the smart list.

 

There is at least one "gottcha"  to watch out for.

If the Series Name happens to return Multiple Series with the same name. For example: "The Flash", and "The Flash (2014)".

At this point the item list will be larger then 1.

We'll have to really put an emphasis on there being Years in the new File Name.

 While I work on this multiple folder problem, I should post a version with those fixes mentioned above.

 

Edited April 4, 2022 by chef

[vaise 307]

30 minutes ago, chef said:

 

I've done some more research into how series are handled in the API,  when they are saved in multiple locations.

When there is more then one location, the API will return an entry for both. Giving a TotalRecordCount: 2.

Side Note: Interestingly enough, each location also gets it's own InternalId. That's good to know for another time. 

Example

{
  "Items": [
    {
      "Name": "Marvel's Daredevil",
      "ServerId": "########################",
      "Id": "120453",
      "Path": "D:\\TV\\Marvel's Daredevil (2015)",
      "RunTimeTicks": 33000000000,
      "IsFolder": true,
      "Type": "Series",
      "AirDays": [],
      "ImageTags": {
        "Primary": "cb788be2bb50cb31ac70ff9cc07c94ce",
        "Logo": "7796e70b20d1857743e1b3a7930fdbcc",
        "Thumb": "d6e69eabb6070776d3a9ff9e305ed1d7"
      },
      "BackdropImageTags": [
        "4978731797448afac5b2a24f8ef5964f",
        "17fd160959fa1e5149e1b19fc38a5089",
        "136f24e54973db9fae9c53bbaa632364",
        "aa47d64a34b499a202d4b987f265b32d",
        "879cb158bb99c8aa0e3817b1a8f27dab"
      ]
    },
    {
      "Name": "Marvel's Daredevil",
      "ServerId": "###############################",
      "Id": "179795",
      "Path": "R:\\TV\\Marvel's Daredevil (2015)",
      "RunTimeTicks": 33000000000,
      "IsFolder": true,
      "Type": "Series",
      "AirDays": [],
      "ImageTags": {
        "Primary": "0b703a1934fd606ccd2619fc96ed6461"
      },
      "BackdropImageTags": [
        "73d7721b7066e365e0dfa403bfaed406",
        "7c5138d0cdb4c02e7e99d3eed6d5ff93",
        "daf929272ebcba13d32ad5f4cc33bdfc"
      ]
    }
  ],
  "TotalRecordCount": 2
}

 

This is how we are going to decide to read the Smart List for sorting or not.

If the Series responds with an Item list larger then 1, we'll know that it has two locations, and we'll check the smart list.

 

There is at least one "gottcha"  to watch out for.

If the Series Name happens to return Multiple Series with the same name. For example: "The Flash", and "The Flash (2014)".

At this point the item list will be larger then 1.

We'll have to really put an emphasis on there being Years in the new File Name.

 While I work on this multiple folder problem, I should post a version with those fixes mentioned above.

 

I think that as long as you are presenting a list to pick the correct show, then that would also take care of shows that are remakes.  I.e Doctor Who, Charmed etc etc.  the first runs don’t have a year, the new runs tend to have a year.  So you are covered.  For the rare times this occurs, I would be happy to keep manually selecting the correct library, as if you do a remember, then one for an old show pops in, it could replace a new one.

ie.

charmed s01e01.  It’s the new one so need to pick the new one, but if remembered, then another charmed s01e01 comes in which is the old original one, it would remember from before and replace the new one not the original one.

[chef 3746]

@vaise

Finally getting to this naming issue sir!

It's taking me a while I apologize.

 

With regards to the renaming of tv episode files adding an unwanted  "0" in front of numbers that don't need it.

I can't seem to reproduce it myself.

Example: 

This is episode 13 for a series I tested on. It doesn't have an unwanted "0".

 

However, if the episode number is below 9, a 0 will show prior to the episode number.

My episode file naming pattern looks as follows:

%sn - %sx%0e - %en.%ext

Result: MySeries - 1x01 - Episode Name.mkv

 

My Season folder pattern doesn't use a trailing "0"

Season %s

Result: Season 1

 

What does your pattern for episode, and seasons look like?

I need to follow the rabbit hole a bit on this one.

 

Edited April 5, 2022 by chef

[vaise 307]

1 hour ago, chef said:

@vaise

Finally getting to this naming issue sir!

It's taking me a while I apologize.

 

With regards to the renaming of tv episode files adding an unwanted  "0" in front of numbers that don't need it.

I can't seem to reproduce it myself.

Example: 

This is episode 13 for a series I tested on. It doesn't have an unwanted "0".

 

However, if the episode number is below 9, a 0 will show prior to the episode number.

My episode file naming pattern looks as follows:

%sn - %sx%0e - %en.%ext

Result: MySeries - 1x01 - Episode Name.mkv

 

My Season folder pattern doesn't use a trailing "0"

Season %s

Result: Season 1

 

What does your pattern for episode, and seasons look like?

I need to follow the rabbit hole a bit on this one.

 

 

Thanks for looking at this.,  I don't use that format however, I want to use SXXEXX as it is all configured that way in my media. 

You can see how yours works as you dont have the leading 0 anyway.

So - If I change to %sn - S0%sE%0e - %en.%ext, it is me adding that 0, and your naming is taking it literally even when the seasons got to 10.

Sonarr gets around this in their naming as I said by using a set length for the season - {Series Title}.S{season:00}E{episode:00}.{Episode Title}

Do you see how they have 'S{season:00}' - that means it is all handles correctly (01,02,03..09, 10, 11) - I have no idea what season 99 changing to season 100 would be with sonarr - but I am a long way off that!!!!!

 

***EDIT ***

I just saw this on your exaplanation - 

I have set it to that and now it should work - was  there all along!!!!!!

 

Edited April 5, 2022 by vaise

[chef 3746]

6 minutes ago, vaise said:

 

Thanks for looking at this.,  I don't use that format however, I want to use SXXEXX as it is all configured that way in my media. 

You can see how yours works as you dont have the leading 0 anyway.

So - If I change to %sn - S0%sE%0e - %en.%ext, it is me adding that 0, and your naming is taking it literally even when the seasons got to 10.

Sonarr gets around this in their naming as I said by using a set length for the season - {Series Title}.S{season:00}E{episode:00}.{Episode Title}

Do you see how they have 'S{season:00}' - that means it is all handles correctly (01,02,03..09, 10, 11) - I have no idea what season 99 changing to season 100 would be with sonarr - but I am a long way off that!!!!!

 

***EDIT ***

I just saw this on your exaplanation - 

I have set it to that and now it should work - was  there all along!!!!!!

 

Does it work? 

[vaise 307]

2 minutes ago, chef said:

Does it work? 

I dont have anything to process as yet.

I expect the multi episode is going to work - now I have it set correctly.......

[Tolerant 70]

I always have files left over from the movie watched folder on my NAS. No idea why Emby won't do a full clean. Any ideas? It all seems setup correctly.