chef 3744 Posted June 3, 2022 Share Posted June 3, 2022 (edited) Auto-Organize PLUS - public beta Built upon the Original Auto Organize plugin, Auto-Organize PLUS is a new version which has some enhancements to help sort new media files into the Emby library. Mobile View: Enhanced Table Layout: with Media Info about the files in the watch folder. Multiple Watch Folder Settings: Existing Library Item/Watch Folder Item Media Info Comparisons: File Status/States: Files in the Watched folder can be in several states depending on user settings. Existing Item Existing Items are items that exist both in the watched folder, and also already exist in the Emby library. Users can click the existing item icon to compare the two items, and help decide if the file should copy over into the library folder, or not. New Resolution New resolutions are items that exist in the library, but may not have the same quality as the item in the watched folder. For example: 1080p, and 2160p. The "New Resolution" status will happen, when "Smart Auto Detection" is turned off for a certain media type. Processing The Processing status will happen when a file is being copied, or moved into the library folder. Success Success happens when a file has completely moved into the library folder with it's new name Failure Failure happens when a file name can not be fully parsed, and the media can not be understood by Auto Organize 2.0 User Input Required (Pending...) If a particular TV Show spans multiple folders within the Emby library, Auto Organize 2.0 will wait for user input to decide where to place the file. This state will may happen with "Smart Auto Detection" turned off, or on. Not Enough Disk Space If a file hasn't enough room to copy into the library folder, because it is full. In Use When a file is being copied into the monitored folder from another source, or is being used by another process. When the file becomes available the status will change. New Media When a file has been added to the monitored folder, and it doesn't exist in the Emby library, but "Smart Auto Detection" has been turned off. The file will enter the "New Item" state if all applicable metadata can be obtained to name it. This will allow the user to sort it when they are ready to. New Edition New Edition will happen if the media item in the watched folder has a different release parsed in it's name. For example: "Theatrical", and "UnCut". Table Sorting: Sort the files by stripping out certain types of media, and only showing others. Also, filter the current displayed items by "Name", "Date", and "Status" Current Issues, and on-going development Plugin configuration Page Headers are currently having issues in Emby. Options are not hidden behind the Header on scroll. This is a core issue that is currently being fixed Smart Matches are currently being enhanced, and the experience with smart matches is currently not what the end result will be. Installation notes: Please shut down the server Remove the "fileorganizer.db" file found in Emby's data folder. Place the plugin DLL in the plugins folder Re-start the server. Clear browsing data DOWNLOAD (stable server only): Emby.AutoOrganize_v1.6.41.43.zip Beta server: Emby.AutoOrganize_v1.6.41.23b.zip (Not updated recently) Edited December 12, 2022 by chef 7 4 Link to comment Share on other sites More sharing options...
gihayes 34 Posted June 3, 2022 Share Posted June 3, 2022 Looking forward to it!! 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 4, 2022 Author Share Posted June 4, 2022 Download has been added to the main post. 1 1 Link to comment Share on other sites More sharing options...
DarKni8 37 Posted June 4, 2022 Share Posted June 4, 2022 @chef You are blessing . Gonna try it now 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 4, 2022 Author Share Posted June 4, 2022 I am currently experiencing some strange behaviour while using the mobile interface, and I'm looking into it now. 1 Link to comment Share on other sites More sharing options...
Cheesegeezer 3086 Posted June 4, 2022 Share Posted June 4, 2022 @chef you little tiger!!! nice work! I shall give it a test drive tomorrow and let you know buddy! 2 hours ago, chef said: I am currently experiencing some strange behaviour while using the mobile interface, and I'm looking into it now. Keith will be happy you’re sorting this out lol 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 4, 2022 Author Share Posted June 4, 2022 I've fixed the strange update behaviour of the mobile cards, and I will put a new link in the main post. It will be version .24 1 Link to comment Share on other sites More sharing options...
gihayes 34 Posted June 4, 2022 Share Posted June 4, 2022 I haven't tried the version in your latest post but the previous version is working great for me!! And subtitles too!! Great man. I tried it with 2 seasons from 2 series 1 movie. Everything processed fine. The only thing it didn't do was clear the txt files from the watch folder. So it did not delete the processed folders. This is such a time saver! Great work! 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 4, 2022 Author Share Posted June 4, 2022 1 hour ago, gihayes said: I haven't tried the version in your latest post but the previous version is working great for me!! And subtitles too!! Great man. I tried it with 2 seasons from 2 series 1 movie. Everything processed fine. The only thing it didn't do was clear the txt files from the watch folder. So it did not delete the processed folders. This is such a time saver! Great work! Thanks for the feedback. Looking forward to more as we continue working on this. Link to comment Share on other sites More sharing options...
chef 3744 Posted June 5, 2022 Author Share Posted June 5, 2022 (edited) Just waiting on some last minute testing results before posting .24. I did come across an interesting thing while using auto organize 2. It has to do with provider data for TV shows that have been released but are marked as "TBA" in the Provider API. Auto organize will sort the file but name it "TBA". This seems fine, as they become accessible in Emby, and they can be viewed. What is interesting is what happens later if you obtain the file again, and the providers have updated their API results. The file will sort it self with the actual episode name, and not overwrite the "TBA" version in the file system. Perhaps records of TBA results be kept somewhere and delt with during a task when the actual name can be resolved by the provider. Edited June 5, 2022 by chef Link to comment Share on other sites More sharing options...
neik 834 Posted June 5, 2022 Share Posted June 5, 2022 (edited) 4 minutes ago, chef said: The file will sort it self with the actual episode name, and not overwrite the "TBA" version in the file system. Wouldn't it make sense to check the Emby library whether the episodes already exists or not, instead of doing that based on file name? It would be more bulletproof, I guess. Edited June 5, 2022 by neik 1 Link to comment Share on other sites More sharing options...
Cheesegeezer 3086 Posted June 5, 2022 Share Posted June 5, 2022 (edited) 1 hour ago, chef said: Just waiting on some last minute testing results before posting .24. I did come across an interesting thing while using auto organize 2. It has to do with provider data for TV shows that have been released but are marked as "TBA" in the Provider API. Auto organize will sort the file but name it "TBA". This seems fine, as they become accessible in Emby, and they can be viewed. What is interesting is what happens later if you obtain the file again, and the providers have updated their API results. The file will sort it self with the actual episode name, and not overwrite the "TBA" version in the file system Which provider are you using? TVDB or TMDB or both? For fall back. I would interigate both to see if one has correct EpName and compare between the 2. 1 hour ago, chef said: Perhaps records of TBA results be kept somewhere and delt with during a task when the actual name can be resolved by the provider. And if both state TBA - store in a Model List in config and put them in the xml(for persistence) to use for Comparing Emby Id against source Id and if they are updated in source…. update ItemName, sortName and also fileName. what do you think? Edited June 5, 2022 by Cheesegeezer Link to comment Share on other sites More sharing options...
chef 3744 Posted June 5, 2022 Author Share Posted June 5, 2022 Yeah, it is when there are new episodes that have just aired, and the provider doesn't have the name yet. This happened with Book of Boba Fett. Each episode was obtained prior to all the providers having the actual data. They were sorted with the TBA episode name. "Book of Boba Fett - s01e02 - TBA.mkv" Thing is, once the episodes are sorted, and the database is cleared, there is no record of those episodes anywhere stating they had "incorrect" file names. Note: Emby, doest really look at the name part of the file, it is more interested in the season and episode codes (s01e02, etc). So in Emby, the episodes are named, but the actual file is not. It still works... Until you obtain the file again in the future. At that point the file is not "Existing" with the actual episode name, it is only existing with the TBA name. I suppose we could do a recursive check on the entire library and look for "TBA" specifically. Or, keep a record of the TBA results. Either way, we will have to attempt to fix them at the beginning of each task. It's a small issue, but something that has to be addressed eventually. Link to comment Share on other sites More sharing options...
Cheesegeezer 3086 Posted June 5, 2022 Share Posted June 5, 2022 3 minutes ago, chef said: Yeah, it is when there are new episodes that have just aired, and the provider doesn't have the name yet. This happened with Book of Boba Fett. Each episode was obtained prior to all the providers having the actual data. They were sorted with the TBA episode name. "Book of Boba Fett - s01e02 - TBA.mkv" Thing is, once the episodes are sorted, and the database is cleared, there is no record of those episodes anywhere stating they had "incorrect" file names. Note: Emby, doest really look at the name part of the file, it is more interested in the season and episode codes (s01e02, etc). So in Emby, the episodes are named, but the actual file is not. It still works... Until you obtain the file again in the future. At that point the file is not "Existing" with the actual episode name, it is only existing with the TBA name. I suppose we could do a recursive check on the entire library and look for "TBA" specifically. Or, keep a record of the TBA results. Either way, we will have to attempt to fix them at the beginning of each task. It's a small issue, but something that has to be addressed eventually. I get ya totally . But I guarantee that user will not like displaying TBA as a title for the episode. So why not call it Episode 1 if it’s TBA but set a bool for false if the name was TBA when scraped. Then you at least know to interrogate the source for correct filename at beginning of each scan 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 5, 2022 Author Share Posted June 5, 2022 2 hours ago, Cheesegeezer said: I get ya totally . But I guarantee that user will not like displaying TBA as a title for the episode. So why not call it Episode 1 if it’s TBA but set a bool for false if the name was TBA when scraped. Then you at least know to interrogate the source for correct filename at beginning of each scan I think you are on to something there. A flag on the result item in the file organizer database. However, the list of "TBA" items must be separate from the "FileOrganizerResults" list in the database, because that can be cleared. It needs to live on, like the Smart Lists do in the database file, so it can be referenced later. (The fileorganizer.db is actually split into two data tables - one for current results, and one for smart lists) Strangely enough emby will name it TBA in the UI if it also is unable to name it at the time of the library scan. That is definitely an interesting core thing. Eventually, (emby must do this during a future library scan) it will correct the name of the item in the library.db. But, the file name itself will not be altered. So it lives in the file system as TBA, and the library.db entry points to that file path. Link to comment Share on other sites More sharing options...
chef 3744 Posted June 6, 2022 Author Share Posted June 6, 2022 I have laid the ground work to handle the TBA issue, by creating a table in the DB called "Smart Path Corrections". Now, when an Item returns "TBA" from the providers, it's Destination Path is saved. The next time the task is run, we'll check to see if the providers have updated their API to have the data we are looking for. If so, we'll fix the file name. If not, we'll try again next time. Simple enough fix for an issue which only happens every once in a while. Link to comment Share on other sites More sharing options...
rbjtech 4170 Posted June 6, 2022 Share Posted June 6, 2022 (edited) 12 hours ago, chef said: I have laid the ground work to handle the TBA issue, by creating a table in the DB called "Smart Path Corrections". Now, when an Item returns "TBA" from the providers, it's Destination Path is saved. The next time the task is run, we'll check to see if the providers have updated their API to have the data we are looking for. If so, we'll fix the file name. If not, we'll try again next time. Simple enough fix for an issue which only happens every once in a while. Smart .. the 'arr's do the same and don't import until it has a non TBA name ... Edited June 6, 2022 by rbjtech 1 Link to comment Share on other sites More sharing options...
kingy444 108 Posted June 12, 2022 Share Posted June 12, 2022 @chef mate some great work - just went through all your commits and compared to my work from when we were trying to improve the core integration. Looks like you have ticked off everything i did except for the TBA issue which has been mentioned by @rbjtech above Thought i would be able to add something here but looks like you have implemented all my stuff already Been too busy on other dev work and didnt realise how evolved you had this one mate. Well done. Let me know if you want me to look at anything - but looks like you have it all in hand 1 Link to comment Share on other sites More sharing options...
chef 3744 Posted June 12, 2022 Author Share Posted June 12, 2022 (edited) 40 minutes ago, kingy444 said: @chef mate some great work - just went through all your commits and compared to my work from when we were trying to improve the core integration. Looks like you have ticked off everything i did except for the TBA issue which has been mentioned by @rbjtech above Thought i would be able to add something here but looks like you have implemented all my stuff already Been too busy on other dev work and didnt realise how evolved you had this one mate. Well done. Let me know if you want me to look at anything - but looks like you have it all in hand I've been able to add a new "corrections" tab. This tab allows you go compare the current file name to what it is supposed to be. If they differ they get added to a list for correction. Below, stranger things was added prior to auto organize and does not have proper file naming. It appears in the list File system: I select the items with the check box for renaming, and press a button. Now the file system is corrected Best way to handle languages, and TBA, and wrong file names. I think. It also makes sure to name the .nfo, thumb.jpg, and any subtitles. Edited June 12, 2022 by chef 1 Link to comment Share on other sites More sharing options...
kingy444 108 Posted June 12, 2022 Share Posted June 12, 2022 10 minutes ago, chef said: Best way to handle languages, and TBA, and wrong file names. I think. Yep i think that works quite well mate - add an option to auto correct too (off by default) This was another thing i thought of doing but hadnt got my head around the architecture enough to implement at the time - again well done mate. At this stage it looks like you have the code in hand - perhaps the only thing i can offer is the logo i submitted to core that was never published I see you have one, but i quite like how mine aligned with the other core plugins. https://github.com/kingy444/Emby.AutoOrganize/blob/master/Emby.AutoOrganize/thumb.jpg Any reason you didnt write a one time 'blow away the old db if you have v1 database' ? Link to comment Share on other sites More sharing options...
Guest phoenixexcinere Posted June 12, 2022 Share Posted June 12, 2022 (edited) Hey, I tried the beta today, but it doesn't work with each of my series. Here is an example from logfile 2022-06-12 19:38:24.270 Info App: Provider results for X Factor Das Unfassbare has 0 results 2022-06-12 19:38:24.270 Warn App: Unable to determine series from /mnt/user/Daten/downloads/X-Factor/X-Factor Serie/X-Factor - Das Unfassbare/Staffel 2/X-Factor Das Unfassbare S02E13 HDTV 720p x264 - iND.mkv 2022-06-12 19:38:24.641 Info App: Extracted information from /mnt/user/Daten/downloads/X-Factor/X-Factor Serie/X-Factor - Das Unfassbare/Staffel 1/X-Factor Das Unfassbare S01E03 HDTV 720p x264 - iND.mkv. Series name X Factor Das Unfassbare, : Season 1, Episode 3. 2022-06-12 19:38:24.642 Info App: Parsing Library for matching Series: X Factor Das Unfassbare 2022-06-12 19:38:24.649 Info App: Parsing Smart List Info to find series name: X Factor Das Unfassbare 2022-06-12 19:38:24.650 Info HttpClient: GET https://api4.thetvdb.com/v4/search?type=series&q=X Factor Das Unfassbare 2022-06-12 19:38:24.745 Info Server: http/1.1 GET http://192.168.178.40:8096/emby/videos/36731/hls1/main/94.ts?PlaySessionId=1a23c1f8eefd4c33afd150ec703f1737. Accept=*/*, Connection=keep-alive, Host=192.168.178.40:8096, User-Agent=Mozilla/5.0 (X11; Linux x86_64; rv:101.0) Gecko/20100101 Firefox/101.0, Accept-Encoding=gzip, deflate, Accept-Language=de,en-US;q=0.7,en;q=0.3, Referer=http://192.168.178.40:8096/web/index.html 2022-06-12 19:38:24.761 Info Server: http/1.1 Response 200 to host1. Time: 17ms. http://192.168.178.40:8096/emby/videos/36731/hls1/main/94.ts?PlaySessionId=1a23c1f8eefd4c33afd150ec703f1737 2022-06-12 19:38:24.800 Error ProviderManager: Error searching *** Error Report *** Version: 4.7.2.0 Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3 Operating system: Linux version 5.15.43-Unraid (root@Develop) (gcc (GCC) 11.2.0, GNU ld version 2.37-slack15) #1 SMP Wed May 25 11:54:05 PDT 2022 Framework: .NET 6.0.2 OS/Process: x64/x64 Runtime: system/System.Private.CoreLib.dll Processor count: 4 Data path: /config Application path: /system MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: TooManyRequests at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod) at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod) at Tvdb.EntryPoint.GetResponse(HttpRequestOptions request, String method) at Tvdb.TvdbBaseProvider`2.FindItemInternal(String searchType, String name, String language, Nullable`1 year, CancellationToken cancellationToken) at Tvdb.TvdbBaseProvider`2.FindItem(TLookupInfoType searchInfo, String searchType, IDirectoryService directoryService, CancellationToken cancellationToken) at Tvdb.TvdbSeriesProvider.GetSearchResults(SeriesInfo searchInfo, CancellationToken cancellationToken) at Emby.Providers.Manager.ProviderManager.GetSearchResults[TLookupType](IRemoteSearchProvider`1 provider, TLookupType searchInfo, CancellationToken cancellationToken) at Emby.Providers.Manager.ProviderManager.GetRemoteSearchResults[TItemType,TLookupType](RemoteSearchQuery`1 searchInfo, BaseItem referenceItem, CancellationToken cancellationToken) Source: Emby.Server.Implementations TargetSite: Void MoveNext() Edited June 12, 2022 by phoenixexcinere Link to comment Share on other sites More sharing options...
chef 3744 Posted June 12, 2022 Author Share Posted June 12, 2022 3 hours ago, kingy444 said: Yep i think that works quite well mate - add an option to auto correct too (off by default) This was another thing i thought of doing but hadnt got my head around the architecture enough to implement at the time - again well done mate. At this stage it looks like you have the code in hand - perhaps the only thing i can offer is the logo i submitted to core that was never published I see you have one, but i quite like how mine aligned with the other core plugins. https://github.com/kingy444/Emby.AutoOrganize/blob/master/Emby.AutoOrganize/thumb.jpg Any reason you didnt write a one time 'blow away the old db if you have v1 database' ? Ohhh... I hadn't thought of the db thing. I guess you would just the file version? Link to comment Share on other sites More sharing options...
chef 3744 Posted June 12, 2022 Author Share Posted June 12, 2022 1 hour ago, phoenixexcinere said: Hey, I tried the beta today, but it doesn't work with each of my series. Here is an example from logfile 2022-06-12 19:38:24.270 Info App: Provider results for X Factor Das Unfassbare has 0 results 2022-06-12 19:38:24.270 Warn App: Unable to determine series from /mnt/user/Daten/downloads/X-Factor/X-Factor Serie/X-Factor - Das Unfassbare/Staffel 2/X-Factor Das Unfassbare S02E13 HDTV 720p x264 - iND.mkv 2022-06-12 19:38:24.641 Info App: Extracted information from /mnt/user/Daten/downloads/X-Factor/X-Factor Serie/X-Factor - Das Unfassbare/Staffel 1/X-Factor Das Unfassbare S01E03 HDTV 720p x264 - iND.mkv. Series name X Factor Das Unfassbare, : Season 1, Episode 3. 2022-06-12 19:38:24.642 Info App: Parsing Library for matching Series: X Factor Das Unfassbare 2022-06-12 19:38:24.649 Info App: Parsing Smart List Info to find series name: X Factor Das Unfassbare 2022-06-12 19:38:24.650 Info HttpClient: GET https://api4.thetvdb.com/v4/search?type=series&q=X Factor Das Unfassbare 2022-06-12 19:38:24.745 Info Server: http/1.1 GET http://192.168.178.40:8096/emby/videos/36731/hls1/main/94.ts?PlaySessionId=1a23c1f8eefd4c33afd150ec703f1737. Accept=*/*, Connection=keep-alive, Host=192.168.178.40:8096, User-Agent=Mozilla/5.0 (X11; Linux x86_64; rv:101.0) Gecko/20100101 Firefox/101.0, Accept-Encoding=gzip, deflate, Accept-Language=de,en-US;q=0.7,en;q=0.3, Referer=http://192.168.178.40:8096/web/index.html 2022-06-12 19:38:24.761 Info Server: http/1.1 Response 200 to host1. Time: 17ms. http://192.168.178.40:8096/emby/videos/36731/hls1/main/94.ts?PlaySessionId=1a23c1f8eefd4c33afd150ec703f1737 2022-06-12 19:38:24.800 Error ProviderManager: Error searching *** Error Report *** Version: 4.7.2.0 Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3 Operating system: Linux version 5.15.43-Unraid (root@Develop) (gcc (GCC) 11.2.0, GNU ld version 2.37-slack15) #1 SMP Wed May 25 11:54:05 PDT 2022 Framework: .NET 6.0.2 OS/Process: x64/x64 Runtime: system/System.Private.CoreLib.dll Processor count: 4 Data path: /config Application path: /system MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: TooManyRequests at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod) at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod) at Tvdb.EntryPoint.GetResponse(HttpRequestOptions request, String method) at Tvdb.TvdbBaseProvider`2.FindItemInternal(String searchType, String name, String language, Nullable`1 year, CancellationToken cancellationToken) at Tvdb.TvdbBaseProvider`2.FindItem(TLookupInfoType searchInfo, String searchType, IDirectoryService directoryService, CancellationToken cancellationToken) at Tvdb.TvdbSeriesProvider.GetSearchResults(SeriesInfo searchInfo, CancellationToken cancellationToken) at Emby.Providers.Manager.ProviderManager.GetSearchResults[TLookupType](IRemoteSearchProvider`1 provider, TLookupType searchInfo, CancellationToken cancellationToken) at Emby.Providers.Manager.ProviderManager.GetRemoteSearchResults[TItemType,TLookupType](RemoteSearchQuery`1 searchInfo, BaseItem referenceItem, CancellationToken cancellationToken) Source: Emby.Server.Implementations TargetSite: Void MoveNext() Yes, we believe that this has to do with the providers, but we are currently looking into it. Many thanks! Link to comment Share on other sites More sharing options...
kingy444 108 Posted June 12, 2022 Share Posted June 12, 2022 4 hours ago, chef said: Yes, we believe that this has to do with the providers, but we are currently looking into it. The original plug-in was returning the same issue so i don’t think it’s your code I am curious with the apiv4 for tvdb did emby get a commercial token ? Hadn’t seen anything much about v4 but perhaps it’s related to that? 1 Link to comment Share on other sites More sharing options...
Luke 36884 Posted June 12, 2022 Share Posted June 12, 2022 26 minutes ago, kingy444 said: The original plug-in was returning the same issue so i don’t think it’s your code I am curious with the apiv4 for tvdb did emby get a commercial token ? Hadn’t seen anything much about v4 but perhaps it’s related to that? Yes we did. 2 Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now