Jump to content

How to find duplicates?


brallor

Recommended Posts

brallor

Hi

 

I have a large library of files and i want to know if there is any addon or trick to find duplicates of movies and tv episodes?

  • Like 1
Link to comment
Share on other sites

CaseyP

This is something that has been asked about and there never seems to be a good answer...

 

In theory you could use Reports to show all episodes, start it collapsed and organized by episode title.. most entries will now end with" : 1 " at the end of the entry.. if two shows have the same title the entry will show have " : 2 " (or more depending on the episodes that share that title) at the end.

 

You could then probably figure out a way to export to excel.. or start by exporting to excel and then using excel to better navigate the database.

 

OR 

 

Run Emby and Plex concurrently, they don't interfere with each other's own database but somehow Plex has figured out how to show duplicates years ago. Crazy, I know. 

 

So now I just run both next to each other.. Emby is great for showing me whats missing or upcoming shows.. Plex is great for actually helping me keep it all organized.

 

It sucks but its been two years since I originally switched from Plex to Emby and brought up this concern.... two years later and it's still not easy to just display duplicates... oh and now I'm running two servers to achieve what just one should be able to do. 

Link to comment
Share on other sites

nagetech

This is something that has been asked about and there never seems to be a good answer...

 

In theory you could use Reports to show all episodes, start it collapsed and organized by episode title.. most entries will now end with" : 1 " at the end of the entry.. if two shows have the same title the entry will show have " : 2 " (or more depending on the episodes that share that title) at the end.

 

You could then probably figure out a way to export to excel.. or start by exporting to excel and then using excel to better navigate the database.

 

OR 

 

Run Emby and Plex concurrently, they don't interfere with each other's own database but somehow Plex has figured out how to show duplicates years ago. Crazy, I know. 

 

So now I just run both next to each other.. Emby is great for showing me whats missing or upcoming shows.. Plex is great for actually helping me keep it all organized.

 

It sucks but its been two years since I originally switched from Plex to Emby and brought up this concern.... two years later and it's still not easy to just display duplicates... oh and now I'm running two servers to achieve what just one should be able to do. 

Howdy! I was actually trying to follow your first bit of advice. I fire up the reports and organize the episodes by name, but I feel a bit stupid. I am not following the part where you said "ends with :1" Where exactly am I suppose to see that? Curious to see if this helps me identify duplicate TV episodes.

 

I've noticed a few times in one season folder for a show I may have 01x01.mkv and 01x01.mp4 as emby auto organized it for me. I had assumed that the emby auto organize would not copy over a new episode if it already was present, turns out if the new file name's extension is different from what is already there, emby just goes right on ahead and moves it. Bummer for me, so trying to see how to find all the goof ups in the collection.

 

Thanks!

Link to comment
Share on other sites

CaseyP

As an example for movies:

 

Go to reports. Make sure 'Media' and 'Movies' are selected in the drop down menu above the dataset. 

 

Click the three vertical dots to the right of the drop down menus. On the panel that opens, use the 'Grouping' drop down menu to group by 'Name' 

 

Then check the box, 'Start Collapsed'

 

You can also select a higher count for the Page Size so you can see all your movies. 

 

Now, In the data set, you'll see the movie title along with a number of how many are in that grouping. 

 

For example, my first movie is '21 Jump Street' with the settings above, that entry reads, "21 Jump Street : 1"

 

Since the end has a "space colon space number' : 1 it signifies there is only 1 movie called 21 Jump Street in the library.

 

Which means if you now use your browsers Find option, usually Ctrl+F, you can search for " : 2" and, if you have the Page Size to be all, youll be able to find all entries that share the same movie title...

 

This also means that movies that share the same title will be found as you go, for instance Ghostbusters.. that entry would be " : 2" if you have both the original and the one from a few years ago.

 

Also, you have to continue to search for a grouping with three possible entries as searching for " : 2" will only find doubles.

 

 

 

TV shows are similar, but you'll start with using the drop down box to show episodes and then follow above.

 

The caveat there is, many shows just use Pilot as their first episode, or if its not named, Episode 1.. so finding duplicates there isn't foolproof.  

 

 

Finally.. to do ALL of that on Plex, from the home screen, select the library to search through, for instance, Movies. Then, above the library showing your movies, where it says "All" click and change that to "Duplicates"... You're done.

Link to comment
Share on other sites

CaseyP

Howdy! I was actually trying to follow your first bit of advice. I fire up the reports and organize the episodes by name, but I feel a bit stupid. I am not following the part where you said "ends with :1" Where exactly am I suppose to see that? Curious to see if this helps me identify duplicate TV episodes.

 

I've noticed a few times in one season folder for a show I may have 01x01.mkv and 01x01.mp4 as emby auto organized it for me. I had assumed that the emby auto organize would not copy over a new episode if it already was present, turns out if the new file name's extension is different from what is already there, emby just goes right on ahead and moves it. Bummer for me, so trying to see how to find all the goof ups in the collection.

 

Thanks!

 

Yeah, Ive had that exact thing happen too. In fact until I reinstalled plex, I wasn't aware that I had 10gbs of duplicate episodes. 

 

The above method may work for you, but in all honesty, save yourself the headache, just install Plex next to Emby and use it for this situation. You don't even have to have Plex running all the time, just let it build the library, find duplicates, turn it off, and make it a monthly routine I suppose. 

 

On a sidenote, one of the main reasons I left Plex was because of the auto organize plugin, along with the ability to show missing episodes. Plex still doesn't show me missing episodes, and Emby's auto organize usually works unless its different file extensions creating duplicates.

 

Then the 4.0 update for Emby came and the ability to filter episodes by missing was removed from the user interface... its now only accessible via reports. 

Link to comment
Share on other sites

Happy2Play

Howdy! I was actually trying to follow your first bit of advice. I fire up the reports and organize the episodes by name, but I feel a bit stupid. I am not following the part where you said "ends with :1" Where exactly am I suppose to see that? Curious to see if this helps me identify duplicate TV episodes.

 

I've noticed a few times in one season folder for a show I may have 01x01.mkv and 01x01.mp4 as emby auto organized it for me. I had assumed that the emby auto organize would not copy over a new episode if it already was present, turns out if the new file name's extension is different from what is already there, emby just goes right on ahead and moves it. Bummer for me, so trying to see how to find all the goof ups in the collection.

 

Thanks!

 

Episodes sorted by series (Reports plugin)

5c93fe6caf2b7_sort.jpg

 

Or you have the EmbyStat server you can see duplicates for movies.  Does appear to be anything for Episode in EmbyStat yet.

 

5c940152b902d_estat.jpg

Edited by Happy2Play
Link to comment
Share on other sites

  • 1 month later...
GtownE

Finally.. to do ALL of that on Plex, from the home screen, select the library to search through, for instance, Movies. Then, above the library showing your movies, where it says "All" click and change that to "Duplicates"... You're done.

 

Can you explain what the final statement you said means?

I followed your entire step by step process until I got to last few sentences

Link to comment
Share on other sites

  • 6 months later...

I have been using the Reports method for this however today for some reason I was getting frequent Emby crashes when using the Report plugin.

 

I then had a look at EmbyStat which does show the duplicate movies however it does not appear to state whether those duplicates have been grouped or not (as the Report method does).

 

So EmbyStat is not useful in this case unless that functionality can be added.

Link to comment
Share on other sites

I have been using the Reports method for this however today for some reason I was getting frequent Emby crashes when using the Report plugin.

@@Jibest, please open a topic about this. Thanks.

Link to comment
Share on other sites

  • 2 months later...
c.michael

I'm fairly new to Emby, but the "finding duplicates" problem is interesting to me as I go through my library and replace low-resolution movies with HD versions.  I have Ruby scripts to manage all of my media and here's what I use to find duplicates: https://gist.github.com/696748ac75912b5769ae94c3d38fe9b0#file-find_duplicate_movies-rb. Hopefully someone will find it useful. And if I can help, let me know.

Link to comment
Share on other sites

I'm fairly new to Emby, but the "finding duplicates" problem is interesting to me as I go through my library and replace low-resolution movies with HD versions.  I have Ruby scripts to manage all of my media and here's what I use to find duplicates: https://gist.github.com/696748ac75912b5769ae94c3d38fe9b0#file-find_duplicate_movies-rb. Hopefully someone will find it useful. And if I can help, let me know.

 

Pretty nifty.. I installed Ruby and started to play around with it.. got a few errors and then remembered.. 

 

This should just be a standard feature in Emby. I still have Plex installed solely so I can check for duplicates with the press of a button. 

 

Finding duplicates, batch metadata editing, I brought both these up years ago and was told it was being looked into or it was on the to-do list... I haven't found a workaround to batch edit meta data yet (adding a tag or genre to multiple items) but since I've got my duplicates workaround, I'm not in any hurry to push the devs on a topic they clearly don't care about. 

Link to comment
Share on other sites

c.michael

Pretty nifty.. I installed Ruby and started to play around with it.. got a few errors and then remembered.. 

 

This should just be a standard feature in Emby. I still have Plex installed solely so I can check for duplicates with the press of a button. 

 

Finding duplicates, batch metadata editing, I brought both these up years ago and was told it was being looked into or it was on the to-do list... I haven't found a workaround to batch edit meta data yet (adding a tag or genre to multiple items) but since I've got my duplicates workaround, I'm not in any hurry to push the devs on a topic they clearly don't care about. 

Sorry you got some errors. I forget that not everyone is a Ruby geek like me.  Plex did make it nice for checking dupes, but I left a lifetime membership there for other reasons (spying, tracking, changing/removing features that I cared about).  And having to login through their server in order to watch my own media over my local network was unacceptable.

 

I'll add some docs to that script, but basically you have to:

 

  • Install Ruby 2.x
  • Install 2 gems
  • Modify the directories array to reflect where your movie libraries are located

Docs are forthcoming on the gist.

 

-C

  • Like 2
Link to comment
Share on other sites

  • 8 months later...
Ronstang

Thanks, now I can't seem to figure out how to generate an API token 

It says to go to Advaned -- Security but I have no Security option there on the dashboard

Edited by Ronstang
Link to comment
Share on other sites

Happy2Play
38 minutes ago, Ronstang said:

Thanks, now I can't seem to figure out how to generate an API token 

It says to go to Advaned -- Security but I have no Security option there on the dashboard

It is directly on the Dashboard navdrawer near the bottom now.

Link to comment
Share on other sites

Happy2Play
2 hours ago, Ronstang said:

I don't see it 

Second from the bottom "API Keys".

Edited by Happy2Play
Link to comment
Share on other sites

Ronstang

Yes, that is not "Security" and I don't know what to do with it.  I'm not a coder or in IT.  I can build computers and do most things with them but I need a little help with this stuff.

I generated an API key but don't know what to do with it or how to use it 

I cut and pasted the key into EmbyStat and then it says to go to advanced - Security and generate a token.....but I don't see where to do that

Edited by Ronstang
Link to comment
Share on other sites

Happy2Play
23 minutes ago, Ronstang said:

Yes, that is not "Security" and I don't know what to do with it.  I'm not a coder or in IT.  I can build computers and do most things with them but I need a little help with this stuff.

I generated an API key but don't know what to do with it or how to use it 

I cut and pasted the key into EmbyStat and then it says to go to advanced - Security and generate a token.....but I don't see where to do that

@reggi needs to update that note as it has moved in Emby UI.

I am guessing your copy and paste may contain a possible trailing space.

Link to comment
Share on other sites

Ronstang

I'm sorry, no matter what I do the API key will not paste into the setup of EmbyStat so I don't know what to do.

Edited by Ronstang
Link to comment
Share on other sites

Ronstang

No matter how I paste the API key into the setup for EmbyStat it keeps telling me I need to generate an API token but I don't know how to do that 

Link to comment
Share on other sites

Happy2Play
13 minutes ago, Ronstang said:

No matter how I paste the API key into the setup for EmbyStat it keeps telling me I need to generate an API token but I don't know how to do that 

Have your tried manually typing the key in.  I just uninstalled removing data, and reinstalling and had no issue applying generated api key.

Link to comment
Share on other sites

Ronstang

Manually typing it worked but I don't understand why pasting it in didn't work.  I did not add a space at the end or front and I tried it many times.....oh well, it's working now.

Thank you .....everyone here is always helpful

Edited by Ronstang
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...