Jump to content

Recommended Posts

Posted

Hey everyone,

I recently encountered a small yet frustrating issue while managing my Emby library: finding duplicate movies. There were two types of duplicates I was dealing with:

  • Two files of the same movie.
  • Emby identifies one movie as another.

To solve this, I decided to create a simple web project that works in any environment. All you need to do is provide your Emby server URL and API key—no need to dive into complex setups or configurations. The tool will scan your libraries for duplicates, and if any are found, it allows you to download a list showing the movie names and their file paths organized by the library.

If you're facing a similar issue, you can try out the tool here: https://imahmud1.github.io/emby-dupe-finder/

For those interested in tinkering with the code or contributing, the project is open-source, and you can find the repo here: https://github.com/imahmud1/emby-dupe-finder

I hope this helps anyone else dealing with duplicate movies in their Emby setup!

Cheers!

image.thumb.png.ff2ef0d2400bd6c34f844deaa93fc195.png
image.thumb.png.6884f641135ee1c79b69e7ee51fdc851.png

  • Like 5
Posted

Very nice, well done!

  • Like 1
  • Agree 1
Posted
4 minutes ago, Abobader said:

Very nice, well done!

Thank you. Though I can't create complex tools or an entire dashboard for general use, I was thinking about whether other small issues like this can be solved that utilizes API. I have already created a simple dashboard that mainly shows streams and the total number of contents.

  • Like 2
Posted

I like this, since it works local too.

Just open the html file, input info.

And Nice list 👍

  • Like 2
Posted
3 hours ago, JayceDK said:

I like this, since it works local too.

Just open the html file, input info.

And Nice list 👍

Glad you liked it, btw you don't even need to download and run this. You can just use the live page link, will work just fine. Since its not sending any data to any hosting the script will work locally. 

Eigeplackter
Posted

Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found,

but after checking the results I'm unsure how effective the tool is:

i.e,:

Duplicate found: Aladdin
  - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019)
  - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992)

Duplicate found: Arielle, die Meerjungfrau
  - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023)
  - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990)

Not just the Disney live-action titles, but other movies too:

Would you be able to add an additional check via movie IDs  when a match by name is given ?

Would get alot of false positive results sorted.

Posted
10 hours ago, Eigeplackter said:

Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found,

but after checking the results I'm unsure how effective the tool is:

i.e,:

Duplicate found: Aladdin
  - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019)
  - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992)

Duplicate found: Arielle, die Meerjungfrau
  - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023)
  - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990)

Not just the Disney live-action titles, but other movies too:

Would you be able to add an additional check via movie IDs  when a match by name is given ?

Would get alot of false positive results sorted.

Say EMBY has a movie called The Abyss (1989). Then there was another movie called The Abyss which was released in 2023. Unfortunately, EMBY stored both of those movies as The Abyss (1989).
This is another type of dupe that goes unnoticed sometimes and needs to be fixed manually or by changing the naming part. But I'll still look into it. 

  • Like 1
Posted

Thanks for sharing this.

  • Like 1
  • 2 weeks later...
Posted

Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you ;)

Error: Failed to fetch libraries

Screenshot2024-08-23at20-38-55EmbyDuplicateFinder.png.514865fb07b74e96a6555892fae07dbe.png

Happy2Play
Posted
2 minutes ago, Houfino said:

Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you ;)

Error: Failed to fetch libraries

Screenshot2024-08-23at20-38-55EmbyDuplicateFinder.png.514865fb07b74e96a6555892fae07dbe.png

Will assume if you bring up browser console you will see CORS errors.  But if you run on same machine as server you can use localhost:port.

Posted
Před 2 minutami Happy2Play řekl:

Předpokládá se, že pokud vyvoláte konzoli prohlížeče, uvidíte chyby CORS. Ale pokud běžíte na stejném počítači jako server, můžete použít localhost:port.

I have synology and how should I enter? 🙂

Happy2Play
Posted (edited)
6 minutes ago, Houfino said:

I have synology and how should I enter? 🙂

Your best option would be browser flag and enter http://address:port if you can not use local host or https.

Look at the first section of this how to for browser flags.

Edited by Happy2Play
Posted
On 8/24/2024 at 12:40 AM, Houfino said:

Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you ;)

Error: Failed to fetch libraries

Screenshot2024-08-23at20-38-55EmbyDuplicateFinder.png.514865fb07b74e96a6555892fae07dbe.png

Sorry for my late reply, this wasn't working because the protocol was missing. Like adding HTTP:// will do the trick. But no worries, I've pushed an update that will handle these cases where if HTTP is missing it will add them on its own. 

Try it out: https://imahmud1.github.io/emby-dupe-finder/

Posted
On 8/23/2024 at 8:52 PM, Happy2Play said:

Your best option would be browser flag and enter http://address:port if you can not use local host or https.

Look at the first section of this how to for browser flags.

Thanks and unfortunately I'm using firefox 😕

Posted
30 minutes ago, Dalpuri said:

Sorry for my late reply, this wasn't working because the protocol was missing. Like adding HTTP:// will do the trick. But no worries, I've pushed an update that will handle these cases where if HTTP is missing it will add them on its own. 

Try it out: https://imahmud1.github.io/emby-dupe-finder/

It doesn't work at all...

Screenshot 2024-08-26 at 17-50-11 Emby Duplicate Finder.png

Happy2Play
Posted
57 minutes ago, Houfino said:

Thanks and unfortunately I'm using firefox 😕

I had no issue running the downloaded version, but yes had to disable protection in Firefox to use the online version as you still get console errors if using LANIP.

image.thumb.png.2e257800266a61e76b659736323f5434.png

image.png.a666628908869cf0f774cd462fa4cf5f.png

image.thumb.png.6fd49c225c9cec59680894c4c474bd6d.png

 

Localhost worked without issue assuming you are browsing on the same machine as Emby,

 

  • Thanks 1
Posted
5 minutes ago, Happy2Play said:

I had no issue running the downloaded version, but yes had to disable protection in Firefox to use the online version as you still get console errors if using LANIP.

image.thumb.png.2e257800266a61e76b659736323f5434.png

image.png.a666628908869cf0f774cd462fa4cf5f.png

image.thumb.png.6fd49c225c9cec59680894c4c474bd6d.png

 

Localhost worked without issue assuming you are browsing on the same machine as Emby,

 

Thank you very much!! It worked 🙂 🙂

  • 2 months later...
Posted (edited)
Dne 8/10/2024 ve 22:04 Eigeplackter řekl:

Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found,

but after checking the results I'm unsure how effective the tool is:

i.e,:

Duplicate found: Aladdin
  - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019)
  - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992)

Duplicate found: Arielle, die Meerjungfrau
  - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023)
  - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990)

Not just the Disney live-action titles, but other movies too:

Would you be able to add an additional check via movie IDs  when a match by name is given ?

Would get alot of false positive results sorted.

 

He is right...Needs an ID fix...Better ID than the name... 😞

@Dalpuri : Can you fix it? Thank you

 

Edited by Houfino
  • 1 month later...
nopenope12345
Posted

@Dalpuri

This tool is great! I've been attempting to rework the script to do this for tv shows because I have been having issues with duplicate episodes. I need to look more into the api calls though to get something working. No luck yet!

  • 2 weeks later...
RLanger7228
Posted

I was pointed to this solution for another problem, but I am also having issues getting it to run.  I am using Safari (Version 17.6 (19618.3.11.11.5)), on an M1 MacBook Pro (MacOS: 14.6.1 (23G93)).  Emby (4.8.10.0) is running on a Synology NAS DS 718+ (DSM 7.2.2-72806 Update 2), directly connected to the router I am connecting to via WiFi.  Using the online version first, I found my API key under Emby Settings:Advanced>API Keys, then double-clicked to copy it, then pasted it into the Dup-Finder field.  I first used the WAN IP address (from Emby Settings>Dashboard), since others mentioned having problems using a local IP address. I tried it directly (SS 1), but got the error: "URL is not valid or contains user credentials".  I also tried it using the displayed template (SS 2) and got a different error (SS 3): "Load Failed".  I tried with HTTP and HTTPS and got the same errors.  I also tried with the In-home (LAN) address from the dashboard, and with both HTTP & HTTPS, but got the same errors again.  

I tried downloading the Dup-Finder through the provided link.  It downloaded to my laptop, so I created a folder on my NAS & uploaded the files.  I can't seem to find a way to run it directly from the NAS.  Double-clicking the index.html file just downloads it back to my laptop.  However, when I just ran it from the downloaded folder on my laptop, it was able to run (SS 4) (yay!!).  BTW, the format that worked was simply  HTTP://my local address:Port -- http://192.168.1.154:8096

The file I was able to download showed that the duplicate movies were all different versions of the same story: Live vs. animated, original vs. remake, etc.  It did not seem to find any mis-identified moves (which was the original problem that brought me here), but that may just mean I have already sorted out all the ones that were mis-identified.

I wrote this as I was working through the problem with various iterations.  Even though I answered my own question, I will post it anyway, since the details of my meanderings may be helpful to other users.  Thank you for the tool.  It will likely prove helpful going forward.

Screenshot 2024-12-25 at 9.44.26 AM.png

Screenshot 2024-12-25 at 9.38.09 AM.png

Screenshot 2024-12-25 at 9.44.26 AM.png

Screenshot 2024-12-25 at 10.52.47 AM.png

  • Thanks 1
RLanger7228
Posted

QUESTION:  If I have downloaded the Dup-Finder & am using it only on my system, is there an easy edit to get the IP address & API key "hardwired" into the script so it just runs without asking for them?  I tried looking at the code with TextEdit, but it was not intuitive for me with my only experience programming in BASIC (the first computer program I ever wrote was in 1975, on punch cards & haven't successfully written any new code for at least 10-20 years, though I tried to dabble in SWIFT about 5 years ago).

nopenope12345
Posted
37 minutes ago, RLanger7228 said:

QUESTION:  If I have downloaded the Dup-Finder & am using it only on my system, is there an easy edit to get the IP address & API key "hardwired" into the script so it just runs without asking for them?  I tried looking at the code with TextEdit, but it was not intuitive for me with my only experience programming in BASIC (the first computer program I ever wrote was in 1975, on punch cards & haven't successfully written any new code for at least 10-20 years, though I tried to dabble in SWIFT about 5 years ago).

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Emby Duplicate Finder</title>
    <link rel="stylesheet" href="styles.css">
    <link href="https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;700&display=swap" rel="stylesheet">
</head>
<body>
    <div class="container">
        <h1>Emby Duplicate Finder</h1>
        <div class="input-group">
            <input value="<yourURL>" type="text" id="embyServerUrl" placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096">
            <input value="<yourAPIKey>" type="text" id="apiKey" placeholder="API Key">
            <button onclick="findDuplicates()">Find Duplicate Movies</button>
        </div>
        <div id="results"></div>
    </div>
    <script src="script.js"></script>
    <div id="loading-overlay" class="hidden">
        <div class="spinner"></div>
        <p>Processing...</p>
    </div>
</body>
</html>

Change your included index.html page to what I posted and you should be set! Obviously change the values :)

  • Like 1
RLanger7228
Posted (edited)

Thank you!!  ♥️

I assume the only values I need to change are: <yourURL> & <yourAPIkey> in:

9 hours ago, nopenope12345 said:
            <input value="<yourURL>" type="text" id="embyServerUrl" placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096">
            <input value="<yourAPIKey>" type="text" id="apiKey" placeholder="API Key">

 

I copied your code above & pasted it into Text Edit, changed the URL & API, saved it as an *.RTF file (required by Text Edit), then renamed it into *.html using Finder.  It seemed to run OK, but there is extraneous code on the left side of the browser screen, as seen in the SS.  I verified that my code looked identical to yours (except for URL & API added).  I copied what I wrote & pasted it below to see if I missed anything (replacing some of the actual API Key with 5 dash segments, for security purposes).

Perhaps Text Edit adds some sort of hidden header code in an *.RTF file that is being mis-interpreted in the *.html execution?

 

Code I wrote:

<!DOCTYPE html>

<html lang="en">

<head>

<meta charset="UTF-8">

<meta name="viewport"content="width=device-width, initial-scale=1.0">

<title>Emby Duplicate Finder</title>

<link rel="stylesheet"href="styles.css">

<link href="https://fonts.googleapis.com/css2?family=Roboto:wght@300; 400; 700&display=swap"  rel="stylesheet">

</head>

<body>

<div class="container">

<h1>Emby Duplicate Finder</h1>

<div class="input-group">

<input value="http://192.168.1.154:8096"  type="text"  id="embyServerUrl"  placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096">

<input value="99cb-----85d4-----06949-----ed3"  type="text"  id="apiKey"  placeholder="API Key">

<button onclick="findDuplicates()">Find Duplicate Movies</button>

</div>

<div id="results"></div>

</div>

<script src="script.js"></script>

<div id="loading-overlay" class="hidden">

<div class="spinner"></div>

<p>Processing...</p>

</div>

</body>

</html>

 

Screenshot 2024-12-28 at 8.10.53 AM.png

Edited by RLanger7228
Accidentally posted before I was finished
nopenope12345
Posted

@RLanger7228

 

Open the original index.html directly with an editor like notepad++ and save it as an .html file. The RTF file is going to introduce extra junk in the file.

RLanger7228
Posted

Thank you.  You are certainly right about the extra junk from TextEdit in an RTF file.  However, I don't have Note Pad on my M1 MacBook (& got a "can't run power PC apps" error when I tried to open it in SpotLight).  

When I tried to open a copy of the readme.md file, it tried to open Xcode, which I haven't used in so long that I don't even have an active copy on my current laptop.  So, I exported a copy of my edited Index.html file to my old laptop (where Xcode IS still installed).  When I opened the file in Xcode, I saw the header info before the actual code.  When I deleted it (and the end of line slashes), saved & exported it back to my current laptop, it now runs cleanly. 😀

I had also tried opening it with MS Word (apple version), but it likely just interpreted the RTF header info without displaying it.  

I always thought TextEdit was the simplest, "cleanest" file editor, but I learned something today - RTF is not the same as TXT.  When I went into the settings of TextEdit, I saw that there IS an option to "Display HTML files as HTML code instead of formatted text".  Had I used that option, I could have just edited & saved the code directly.  Without that option checked, opening the file in TextEdit just displays the executed code.

Alternatively, (just as a goof) when I added .TXT to the end (forcing it to be recognized as a simple text file), I saw the code you wrote and the RTF header junk, so I could have edited it that way, also.  TXT files are "clean" & show all data within, but RTF files have extra header/formatting info that gets interpreted without being displayed.

Ya learn somethin every day...

 

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...