Jump to content


Photo

Emby Sweeper


  • Please log in to reply
34 replies to this topic

#1 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 21 February 2020 - 12:48 PM

Kind of wish I found this forum and this topic before I started down this road myself, but anyway I finished my tool and so if there's any interest I can throw the docker on docker hub (that's how I'm calling it in my env) or I can make the executable jar available on github or something (it's a java command line app).

 

Basically it does what the other tool does: finds items in your emby library and deletes them.  I'm using it to cleanup watched tv episodes for the other users in the house who aren't nearly as diligent as me about cleaning things up.

 

Anyway, not sure if this provides any more value than the other existing tool (I probably would have just used it if i came looking here first), but if anyone is interested let me know and I'll package it up and make it available.

 

EDIT: Removed examples and command line options, etc.  Instead, refer to the project wiki, which will be kept up to date.


Edited by Slugger, 22 March 2020 - 08:33 PM.


#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 153199 posts
  • Local time: 11:16 PM

Posted 21 February 2020 - 01:26 PM

Cool thanks for sharing !

#3 sfatula OFFLINE  

sfatula

    Advanced Member

  • Members
  • 666 posts
  • Local time: 10:16 PM
  • LocationCalera, OK

Posted 21 February 2020 - 01:33 PM

I'd be interested in the source. This one appears to have a few extra options. 


Edited by sfatula, 21 February 2020 - 01:34 PM.


#4 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 22 February 2020 - 01:03 PM

I've thrown my source code up in my github:

 

https://github.com/Slugger/embysweeper

 

When I have another free morning I'll work on uploading a compiled jar and getting the docker image I'm using into docker hub.  But until then, all you need to build the jar is jdk8+, clone the repo and run gradlew shadowJar

 

With the generated jar, run

 

java -jar embysweeper-shadow-1.0-SNAPSHOT.jar



#5 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 24 February 2020 - 06:10 PM

A docker image is now available:

 

https://hub.docker.c...ger/embysweeper

 

I've changed my env to start using this image so it will be kept up to date.  To run the example from post #1 above using the docker image:

 

EDIT: Examples moved to wiki


Edited by Slugger, 22 March 2020 - 09:02 PM.


#6 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 26 February 2020 - 06:06 PM

Binary releases now being published on my github:

 

https://github.com/S...weeper/releases

 

The binary (jar) will run on Java 8+, tested on OpenJDK 8.

 

Each binary release will also publish a docker image to docker hub tagged the same.  The latest tag in docker hub will always be built from the master branch.



#7 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 10 March 2020 - 06:38 AM

This is brilliant and looks functionally more capable than the other tool as it can distinguish between a series being a favourite and episodes being favourites!

 

Quick question... to run this what are the pros and cons for running the docker image vs. the binary (jar)?

 

My server is on windows 10 so not sure how relevant that is. I'm trying to keep my server as lightweight as possible. The other tool requires python (which I can now get rid of I think) and your tool requires either Java or Docker (presumably with some version of java preinstalled in your embysweeper docker image) and I'm not sure which is the leanest. I have neither already installed.

 

My understanding is that java consumes less resources. Point 3 in this link indicates that too....

 

https://www.reddit.c..._when_jars_are/

 

I suppose that makes sense as it's just "Java" as opposed to "Docker + Java".

 

Does my logic make any sense? Am I missing something?

 

Thanks loads!!!!


Edited by Kurosh, 10 March 2020 - 06:52 AM.


#8 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 10 March 2020 - 07:52 AM

Yup, you pretty much got it. On windows just installing java directly is by far the lighter option.

#9 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 11 March 2020 - 06:52 PM

Hey,

 

Thanks for this fantastic script. It appears to be working quite well apart from it doesn't appear to delete as much as I'd expect. Is there any way to see the last watched date of an episode? I've turned on trace logging and all I can see is that a particular series is not marked as a favourite. I've a sneaky suspicion someone marked the whole series as watched (within the last 7 days). Maybe in a weeks time it'll delete loads more :)



#10 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 11 March 2020 - 07:30 PM

It will print the last watched date for things it deletes, but the trace logging won't print last watched for things it doesn't plan on deleting.  If you open a ticket I could probably add that next time I get inspired to work on the code.

 

In the meantime, if you enable debug logging for the 3rd party libs via --lib-log-level then the raw api calls and responses to the emby server will be logged.  The raw responses should show the last watched date for all items returned.  It won't be formatted pretty or anything but it should at least allow you to figure out why things you expect to be deleted aren't.



#11 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 12 March 2020 - 05:01 AM

Thanks for promptly responding! Appreciate it.

 

This is the only line in the DEBUG log that mentions the programme (The Real Housewives of Potomac - don't judge, it's the wife's choice, I promise!).

 

08:54:10.841 [main] DEBUG org.apache.http.wire - << "{"Items":[{"Name":"The Real Housewives of Potomac","ServerId":"a7c462c843024cf1991beca8f6574618","Id":"679","RunTimeTicks":27000000000,"IsFolder":true,"Type":"Series","UserData":{"UnplayedItemCount":0,"PlaybackPositionTicks":0,"PlayCount":0,"IsFavorite":false,"Played":true},"AirDays":[],"ImageTags":{"Primary":"b0ec200562425217b9ff51dd08ab87f1","Banner":"2ba9cd9c948eda25009840a43b821159"},"BackdropImageTags":["80aa3c658181875c81b45a60dc52ee4e"]}],"TotalRecordCount":1}"

Does this tell you anything?



#12 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 12 March 2020 - 09:20 AM

So that's the output of a series folder.  The only thing I look at for series folders is whether or not the series is marked as a fav -- this one is not:

 

"UserData":{"UnplayedItemCount":0,"PlaybackPositionTicks":0,"PlayCount":0,"IsFavorite":false,"Played":true}

 

I then cache that response and when I encounter any episode of this series, I first look at this value, when --ignore-fav-series option is used, to determine if I need to do any other checks.  All the other checks are done on the response of the episode item itself.  The last watched date comes from episode items when they've been marked as watched.



#13 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 12 March 2020 - 09:31 AM

Ah I see. So no way of understanding why something wasn't deleted. Never mind, I'll wait a week and see if it deletes :)



#14 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 12 March 2020 - 12:33 PM

Yes, but would need the full debug output from the api calls and all of the command options you used (less id/pwd -- note that when you log the raw api web calls, your id/pwd will be exposed in those logs as well).  Basically, there's four ways something gets filtered out and not deleted:

 

* The episode is located in an excluded library (-b options)

* The episode was filtered out by emby (--filter options)

* The episode watch date was not old enough (--min-age-days; default is 7 when not specified)

* The series is marked as a fav (--ignore-fav-series)

 

Any of those reasons would prevent an episode from being deleted.  With --lib-log-level=DEBUG, --log-level=TRACE, the list of command line args used and an episode name, it could be determined which of the 4 reasons an episode was filtered out.



#15 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 13 March 2020 - 04:37 AM

Here's a log attached with --lib-log-level=DEBUG and --log-level=TRACE arguments set and I've obscured the id/pwd too as you suggested. I can't see as to why many series aren't getting deleted, my guess is the LastPlayedDate is less than a week or possibly there isn't a LastPlayedDate - I've seen this mentioned in the other Emby Media Clean tool forum, not sure on the scenario as to when this happens.

 

For example, "Real Housewives of Potomac" isn't getting deleted, I can't see anything other than it isn't set as a series favourite, can you see anything more? :|

 

Thanks for your time.

Attached Files


Edited by Kurosh, 13 March 2020 - 04:41 AM.


#16 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 13 March 2020 - 09:33 AM

Defintely something to do with the last played date.  The episodes are found in the lib then after the emby filters are applied, they all disappear.  I don't dump enough info out to the logs in trace mode to pinpoint the exact reason, but next time I get a chance to play with the code I'll add some more logging to help get a definitive answer.  It would appear your emby server isn't returning the last played date as expected.  I thought I handled that case, but maybe not.  I'll need the extra trace logging data to know for sure.



#17 Kurosh OFFLINE  

Kurosh

    Advanced Member

  • Members
  • 69 posts
  • Local time: 03:16 AM

Posted 15 March 2020 - 04:50 PM

Did you mean you'll need the extra logging from the code you'll put in next time you have a chance to play with it or did you mean I should set both arguments --lib-log-level and --log-level to TRACE and send a log back here?



#18 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 15 March 2020 - 06:07 PM

The former.  I need to add more logging to the code to get the answers we're after.  The next release will include the extra logging needed.



#19 Slugger OFFLINE  

Slugger

    Member

  • Members
  • 20 posts
  • Local time: 11:16 PM

Posted 19 March 2020 - 09:25 PM

Had some time to implement an audit command in the tool.  The changes are in the master branch and the latest tag in docker hub has these changes.  I'll tag a formal release in a week or so after I've ran it in my env for a little while to make sure there's nothing wrong.  Note, the release version will bump to 2.0 as the command line structure will change.  Take note of the new command line structure (I'll update the first post when I release it).  The next release will include this command and it will look like this:

 

$ /usr/bin/docker run --rm slugger/embysweeper:latest -s 192.168.1.2 -u admin -p password -e Kid1 -e Kid2 -e Wifey -b Movies -b "Home Videos" --ignore-fav-series --filter IsPlayed=true --filter IsFavorite=false audit -f "/data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv"
01:01:26.129 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if /data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv is a valid media file...
01:01:26.222 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - {
    "Name": "Leave a Light On",
    "ServerId": "d8bc58c7fc0d485c95acc69addde3b5d",
    "Id": "13415",
    "RunTimeTicks": 24964930000,
    "IndexNumber": 16,
    "ParentIndexNumber": 16,
    "IsFolder": false,
    "Type": "Episode",
    "ParentLogoItemId": "6338",
    "ParentBackdropItemId": "6338",
    "ParentBackdropImageTags": [
        "b573357b2917fef76ee0b477e940c8b4"
    ],
    "SeriesName": "Grey's Anatomy",
    "SeriesId": "6338",
    "SeasonId": "6343",
    "SeriesPrimaryImageTag": "88ddbceefccb3fad2cdd36c7dd0324aa",
    "SeasonName": "Season 16",
    "ImageTags": {
        "Primary": "90198e5c7158732dd5c4c9b457d1a0f5"
    },
    "BackdropImageTags": [
        
    ],
    "ParentLogoImageTag": "636db2ec9fd5c966c0c1002d902f203a",
    "ParentThumbItemId": "6338",
    "ParentThumbImageTag": "f405c960d3db60efd505e6f05a475bb5",
    "MediaType": "Video"
}
01:01:26.222 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Media file exists in Emby server... onto the next check
01:01:26.223 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if file is excluded by cleanup users...
01:01:26.280 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - This file is visible by one or more users [Wifey]... onto the next check
01:01:26.281 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if the file is excluded by excluded libraries...
01:01:26.317 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - The file is still visible by at least one user... onto the next check
01:01:26.318 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if any of the given filters have excluded the file...
01:01:26.322 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsPlayed=true' filter...
01:01:26.338 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsFavorite=false' filter...
01:01:26.349 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - None of the filters excluded this file from being deleted... onto the next check
01:01:26.350 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if the --ignore-fav-series flag was set...
01:01:26.350 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - It is... checking if any visible user has not marked the series as a fav...
01:01:26.361 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - There is at least one user who has not marked this as a fav, it can still be deleted... onto next check
01:01:26.361 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Last check... check if the item is not old enough yet...
01:01:26.362 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - To be deleted the item must have been watched more than 7 days ago...
01:01:26.384 [main] ERROR com.github.slugger.emby.sweeper.commands.Audit - This file will not be deleted because every specified user last watched it within the last 7 days
 
And just to show what happens when one of the api filters is the reason...
 
$ /usr/bin/docker run --rm slugger/embysweeper:latest -s 192.168.1.2 -u admin -p password -e Kid1 -e Kid2 -e Wifey -b Movies -b "Home Videos" --ignore-fav-series --filter IsPlayed=false --filter IsFavorite=false audit -f "/data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv"
01:11:34.164 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if /data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv is a valid media file...
01:11:34.265 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - {
    "Name": "Leave a Light On",
    "ServerId": "d8bc58c7fc0d485c95acc69addde3b5d",
    "Id": "13415",
    "RunTimeTicks": 24964930000,
    "IndexNumber": 16,
    "ParentIndexNumber": 16,
    "IsFolder": false,
    "Type": "Episode",
    "ParentLogoItemId": "6338",
    "ParentBackdropItemId": "6338",
    "ParentBackdropImageTags": [
        "b573357b2917fef76ee0b477e940c8b4"
    ],
    "SeriesName": "Grey's Anatomy",
    "SeriesId": "6338",
    "SeasonId": "6343",
    "SeriesPrimaryImageTag": "88ddbceefccb3fad2cdd36c7dd0324aa",
    "SeasonName": "Season 16",
    "ImageTags": {
        "Primary": "90198e5c7158732dd5c4c9b457d1a0f5"
    },
    "BackdropImageTags": [
        
    ],
    "ParentLogoImageTag": "636db2ec9fd5c966c0c1002d902f203a",
    "ParentThumbItemId": "6338",
    "ParentThumbImageTag": "f405c960d3db60efd505e6f05a475bb5",
    "MediaType": "Video"
}
01:11:34.265 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Media file exists in Emby server... onto the next check
01:11:34.266 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if file is excluded by cleanup users...
01:11:34.323 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - This file is visible by one or more users [Wifey]... onto the next check
01:11:34.324 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if the file is excluded by excluded libraries...
01:11:34.356 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - The file is still visible by at least one user... onto the next check
01:11:34.356 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if any of the given filters have excluded the file...
01:11:34.359 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsPlayed=false' filter...
01:11:34.371 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsFavorite=false' filter...
01:11:34.382 [main] ERROR com.github.slugger.emby.sweeper.commands.Audit - This file was not deleted because it was excluded by the following filters: [IsPlayed:false]

 

And finally, an example where the file would be deleted if the delete command was used instead of audit:

 

$ /usr/bin/docker run --rm slugger/embysweeper:latest -s 192.168.1.2 -u admin -p password -e Kid1 -e Kid2 -e Wifey -b Movies -b "Home Videos" --ignore-fav-series --filter IsPlayed=true --filter IsFavorite=false --min-age-days 0 audit -f "/data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv"
01:16:16.598 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if /data/tvshows/Wifey/Grey's Anatomy/Grey's Anatomy - S16E16 - Leave a Light On HDTV-720p.mkv is a valid media file...
01:16:16.687 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - {
    "Name": "Leave a Light On",
    "ServerId": "d8bc58c7fc0d485c95acc69addde3b5d",
    "Id": "13415",
    "RunTimeTicks": 24964930000,
    "IndexNumber": 16,
    "ParentIndexNumber": 16,
    "IsFolder": false,
    "Type": "Episode",
    "ParentLogoItemId": "6338",
    "ParentBackdropItemId": "6338",
    "ParentBackdropImageTags": [
        "b573357b2917fef76ee0b477e940c8b4"
    ],
    "SeriesName": "Grey's Anatomy",
    "SeriesId": "6338",
    "SeasonId": "6343",
    "SeriesPrimaryImageTag": "88ddbceefccb3fad2cdd36c7dd0324aa",
    "SeasonName": "Season 16",
    "ImageTags": {
        "Primary": "90198e5c7158732dd5c4c9b457d1a0f5"
    },
    "BackdropImageTags": [
        
    ],
    "ParentLogoImageTag": "636db2ec9fd5c966c0c1002d902f203a",
    "ParentThumbItemId": "6338",
    "ParentThumbImageTag": "f405c960d3db60efd505e6f05a475bb5",
    "MediaType": "Video"
}
01:16:16.687 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Media file exists in Emby server... onto the next check
01:16:16.687 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if file is excluded by cleanup users...
01:16:16.744 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - This file is visible by one or more users [Wifey]... onto the next check
01:16:16.744 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if the file is excluded by excluded libraries...
01:16:16.778 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - The file is still visible by at least one user... onto the next check
01:16:16.778 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if any of the given filters have excluded the file...
01:16:16.781 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsPlayed=true' filter...
01:16:16.794 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking 'IsFavorite=false' filter...
01:16:16.803 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - None of the filters excluded this file from being deleted... onto the next check
01:16:16.804 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Checking if the --ignore-fav-series flag was set...
01:16:16.804 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - It is... checking if any visible user has not marked the series as a fav...
01:16:16.815 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - There is at least one user who has not marked this as a fav, it can still be deleted... onto next check
01:16:16.815 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - Last check... check if the item is not old enough yet...
01:16:16.815 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - To be deleted the item must have been watched more than 0 days ago...
01:16:16.840 [main] INFO com.github.slugger.emby.sweeper.commands.Audit - This file should be deleted by the DELETE action!
 


#20 tropicaljoe OFFLINE  

tropicaljoe

    Member

  • Members
  • 26 posts
  • Local time: 10:16 PM

Posted 20 March 2020 - 03:21 PM

Hi Slugger

Great piece of work!

 

I have been using the docker file but have been unsuccessful in getting the command to exclude a library

 

I have attached the output from running the command. In the file user 'joe' has access to the library "Penzi Movie Collection" and I have excluded this in the command, however it still wants to delete the file. Am I doing something wrong to exclude the  library?. I did the audit as the normal command wanted to delete several files from the library which is shared so I dont want anything deleted

Thanks

Attached Files






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users