Dalpuri 19 Posted August 10, 2024 Posted August 10, 2024 Hey everyone, I recently encountered a small yet frustrating issue while managing my Emby library: finding duplicate movies. There were two types of duplicates I was dealing with: Two files of the same movie. Emby identifies one movie as another. To solve this, I decided to create a simple web project that works in any environment. All you need to do is provide your Emby server URL and API key—no need to dive into complex setups or configurations. The tool will scan your libraries for duplicates, and if any are found, it allows you to download a list showing the movie names and their file paths organized by the library. If you're facing a similar issue, you can try out the tool here: https://imahmud1.github.io/emby-dupe-finder/ For those interested in tinkering with the code or contributing, the project is open-source, and you can find the repo here: https://github.com/imahmud1/emby-dupe-finder I hope this helps anyone else dealing with duplicate movies in their Emby setup! Cheers! 5
Dalpuri 19 Posted August 10, 2024 Author Posted August 10, 2024 4 minutes ago, Abobader said: Very nice, well done! Thank you. Though I can't create complex tools or an entire dashboard for general use, I was thinking about whether other small issues like this can be solved that utilizes API. I have already created a simple dashboard that mainly shows streams and the total number of contents. 2
Neminem 738 Posted August 10, 2024 Posted August 10, 2024 I like this, since it works local too. Just open the html file, input info. And Nice list 2
Dalpuri 19 Posted August 10, 2024 Author Posted August 10, 2024 3 hours ago, JayceDK said: I like this, since it works local too. Just open the html file, input info. And Nice list Glad you liked it, btw you don't even need to download and run this. You can just use the live page link, will work just fine. Since its not sending any data to any hosting the script will work locally.
Eigeplackter 79 Posted August 10, 2024 Posted August 10, 2024 Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found, but after checking the results I'm unsure how effective the tool is: i.e,: Duplicate found: Aladdin - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019) - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992) Duplicate found: Arielle, die Meerjungfrau - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023) - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990) Not just the Disney live-action titles, but other movies too: Would you be able to add an additional check via movie IDs when a match by name is given ? Would get alot of false positive results sorted.
Dalpuri 19 Posted August 11, 2024 Author Posted August 11, 2024 10 hours ago, Eigeplackter said: Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found, but after checking the results I'm unsure how effective the tool is: i.e,: Duplicate found: Aladdin - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019) - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992) Duplicate found: Arielle, die Meerjungfrau - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023) - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990) Not just the Disney live-action titles, but other movies too: Would you be able to add an additional check via movie IDs when a match by name is given ? Would get alot of false positive results sorted. Say EMBY has a movie called The Abyss (1989). Then there was another movie called The Abyss which was released in 2023. Unfortunately, EMBY stored both of those movies as The Abyss (1989). This is another type of dupe that goes unnoticed sometimes and needs to be fixed manually or by changing the naming part. But I'll still look into it. 1
Houfino 31 Posted August 23, 2024 Posted August 23, 2024 Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you Error: Failed to fetch libraries
Happy2Play 9229 Posted August 23, 2024 Posted August 23, 2024 2 minutes ago, Houfino said: Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you Error: Failed to fetch libraries Will assume if you bring up browser console you will see CORS errors. But if you run on same machine as server you can use localhost:port.
Houfino 31 Posted August 23, 2024 Posted August 23, 2024 Před 2 minutami Happy2Play řekl: Předpokládá se, že pokud vyvoláte konzoli prohlížeče, uvidíte chyby CORS. Ale pokud běžíte na stejném počítači jako server, můžete použít localhost:port. I have synology and how should I enter?
Happy2Play 9229 Posted August 23, 2024 Posted August 23, 2024 (edited) 6 minutes ago, Houfino said: I have synology and how should I enter? Your best option would be browser flag and enter http://address:port if you can not use local host or https. Look at the first section of this how to for browser flags. Edited August 23, 2024 by Happy2Play
Dalpuri 19 Posted August 26, 2024 Author Posted August 26, 2024 On 8/24/2024 at 12:40 AM, Houfino said: Thanks and I don't know how it works... I tried to enter local IP + API.. It doesn't work and error..Am I doing something wrong? How do I enter correctly? Thank you Error: Failed to fetch libraries Sorry for my late reply, this wasn't working because the protocol was missing. Like adding HTTP:// will do the trick. But no worries, I've pushed an update that will handle these cases where if HTTP is missing it will add them on its own. Try it out: https://imahmud1.github.io/emby-dupe-finder/
Houfino 31 Posted August 26, 2024 Posted August 26, 2024 On 8/23/2024 at 8:52 PM, Happy2Play said: Your best option would be browser flag and enter http://address:port if you can not use local host or https. Look at the first section of this how to for browser flags. Thanks and unfortunately I'm using firefox
Houfino 31 Posted August 26, 2024 Posted August 26, 2024 30 minutes ago, Dalpuri said: Sorry for my late reply, this wasn't working because the protocol was missing. Like adding HTTP:// will do the trick. But no worries, I've pushed an update that will handle these cases where if HTTP is missing it will add them on its own. Try it out: https://imahmud1.github.io/emby-dupe-finder/ It doesn't work at all...
Happy2Play 9229 Posted August 26, 2024 Posted August 26, 2024 57 minutes ago, Houfino said: Thanks and unfortunately I'm using firefox I had no issue running the downloaded version, but yes had to disable protection in Firefox to use the online version as you still get console errors if using LANIP. Localhost worked without issue assuming you are browsing on the same machine as Emby, 1
Houfino 31 Posted August 26, 2024 Posted August 26, 2024 5 minutes ago, Happy2Play said: I had no issue running the downloaded version, but yes had to disable protection in Firefox to use the online version as you still get console errors if using LANIP. Localhost worked without issue assuming you are browsing on the same machine as Emby, Thank you very much!! It worked
Houfino 31 Posted November 11, 2024 Posted November 11, 2024 (edited) Dne 8/10/2024 ve 22:04 Eigeplackter řekl: Hmmm it seems as the tool only compares filenames. I was astonished, when 54 duplicates were found, but after checking the results I'm unsure how effective the tool is: i.e,: Duplicate found: Aladdin - /volume1/video/Filme/Aladdin (2019)/Aladdin (2019) 1080p DTS.mkv (2019) - /volume1/video/Filme/Aladdin (1992)/Aladdin (1992) 1080p DTS.mkv (1992) Duplicate found: Arielle, die Meerjungfrau - /volume1/video/Filme/Arielle, die Meerjungfrau (2023)/Arielle, die Meerjungfrau (2023) 2160p EAC3.mkv (2023) - /volume1/video/Filme/Arielle, die Meerjungfrau (1990)/Arielle, die Meerjungfrau (1990) 1080p DTS.mkv (1990) Not just the Disney live-action titles, but other movies too: Would you be able to add an additional check via movie IDs when a match by name is given ? Would get alot of false positive results sorted. He is right...Needs an ID fix...Better ID than the name... @Dalpuri : Can you fix it? Thank you Edited November 11, 2024 by Houfino
nopenope12345 5 Posted December 16, 2024 Posted December 16, 2024 @Dalpuri This tool is great! I've been attempting to rework the script to do this for tv shows because I have been having issues with duplicate episodes. I need to look more into the api calls though to get something working. No luck yet!
RLanger7228 44 Posted December 25, 2024 Posted December 25, 2024 I was pointed to this solution for another problem, but I am also having issues getting it to run. I am using Safari (Version 17.6 (19618.3.11.11.5)), on an M1 MacBook Pro (MacOS: 14.6.1 (23G93)). Emby (4.8.10.0) is running on a Synology NAS DS 718+ (DSM 7.2.2-72806 Update 2), directly connected to the router I am connecting to via WiFi. Using the online version first, I found my API key under Emby Settings:Advanced>API Keys, then double-clicked to copy it, then pasted it into the Dup-Finder field. I first used the WAN IP address (from Emby Settings>Dashboard), since others mentioned having problems using a local IP address. I tried it directly (SS 1), but got the error: "URL is not valid or contains user credentials". I also tried it using the displayed template (SS 2) and got a different error (SS 3): "Load Failed". I tried with HTTP and HTTPS and got the same errors. I also tried with the In-home (LAN) address from the dashboard, and with both HTTP & HTTPS, but got the same errors again. I tried downloading the Dup-Finder through the provided link. It downloaded to my laptop, so I created a folder on my NAS & uploaded the files. I can't seem to find a way to run it directly from the NAS. Double-clicking the index.html file just downloads it back to my laptop. However, when I just ran it from the downloaded folder on my laptop, it was able to run (SS 4) (yay!!). BTW, the format that worked was simply HTTP://my local address:Port -- http://192.168.1.154:8096 The file I was able to download showed that the duplicate movies were all different versions of the same story: Live vs. animated, original vs. remake, etc. It did not seem to find any mis-identified moves (which was the original problem that brought me here), but that may just mean I have already sorted out all the ones that were mis-identified. I wrote this as I was working through the problem with various iterations. Even though I answered my own question, I will post it anyway, since the details of my meanderings may be helpful to other users. Thank you for the tool. It will likely prove helpful going forward. 1
RLanger7228 44 Posted December 28, 2024 Posted December 28, 2024 QUESTION: If I have downloaded the Dup-Finder & am using it only on my system, is there an easy edit to get the IP address & API key "hardwired" into the script so it just runs without asking for them? I tried looking at the code with TextEdit, but it was not intuitive for me with my only experience programming in BASIC (the first computer program I ever wrote was in 1975, on punch cards & haven't successfully written any new code for at least 10-20 years, though I tried to dabble in SWIFT about 5 years ago).
nopenope12345 5 Posted December 28, 2024 Posted December 28, 2024 37 minutes ago, RLanger7228 said: QUESTION: If I have downloaded the Dup-Finder & am using it only on my system, is there an easy edit to get the IP address & API key "hardwired" into the script so it just runs without asking for them? I tried looking at the code with TextEdit, but it was not intuitive for me with my only experience programming in BASIC (the first computer program I ever wrote was in 1975, on punch cards & haven't successfully written any new code for at least 10-20 years, though I tried to dabble in SWIFT about 5 years ago). <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Emby Duplicate Finder</title> <link rel="stylesheet" href="styles.css"> <link href="https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;700&display=swap" rel="stylesheet"> </head> <body> <div class="container"> <h1>Emby Duplicate Finder</h1> <div class="input-group"> <input value="<yourURL>" type="text" id="embyServerUrl" placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096"> <input value="<yourAPIKey>" type="text" id="apiKey" placeholder="API Key"> <button onclick="findDuplicates()">Find Duplicate Movies</button> </div> <div id="results"></div> </div> <script src="script.js"></script> <div id="loading-overlay" class="hidden"> <div class="spinner"></div> <p>Processing...</p> </div> </body> </html> Change your included index.html page to what I posted and you should be set! Obviously change the values 1
RLanger7228 44 Posted December 28, 2024 Posted December 28, 2024 (edited) Thank you!! I assume the only values I need to change are: <yourURL> & <yourAPIkey> in: 9 hours ago, nopenope12345 said: <input value="<yourURL>" type="text" id="embyServerUrl" placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096"> <input value="<yourAPIKey>" type="text" id="apiKey" placeholder="API Key"> I copied your code above & pasted it into Text Edit, changed the URL & API, saved it as an *.RTF file (required by Text Edit), then renamed it into *.html using Finder. It seemed to run OK, but there is extraneous code on the left side of the browser screen, as seen in the SS. I verified that my code looked identical to yours (except for URL & API added). I copied what I wrote & pasted it below to see if I missed anything (replacing some of the actual API Key with 5 dash segments, for security purposes). Perhaps Text Edit adds some sort of hidden header code in an *.RTF file that is being mis-interpreted in the *.html execution? Code I wrote: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport"content="width=device-width, initial-scale=1.0"> <title>Emby Duplicate Finder</title> <link rel="stylesheet"href="styles.css"> <link href="https://fonts.googleapis.com/css2?family=Roboto:wght@300; 400; 700&display=swap" rel="stylesheet"> </head> <body> <div class="container"> <h1>Emby Duplicate Finder</h1> <div class="input-group"> <input value="http://192.168.1.154:8096" type="text" id="embyServerUrl" placeholder="Emby Server URL www.domain.com || IP:8096 || locahost:8096"> <input value="99cb-----85d4-----06949-----ed3" type="text" id="apiKey" placeholder="API Key"> <button onclick="findDuplicates()">Find Duplicate Movies</button> </div> <div id="results"></div> </div> <script src="script.js"></script> <div id="loading-overlay" class="hidden"> <div class="spinner"></div> <p>Processing...</p> </div> </body> </html> Edited December 28, 2024 by RLanger7228 Accidentally posted before I was finished
nopenope12345 5 Posted December 28, 2024 Posted December 28, 2024 @RLanger7228 Open the original index.html directly with an editor like notepad++ and save it as an .html file. The RTF file is going to introduce extra junk in the file.
RLanger7228 44 Posted December 28, 2024 Posted December 28, 2024 Thank you. You are certainly right about the extra junk from TextEdit in an RTF file. However, I don't have Note Pad on my M1 MacBook (& got a "can't run power PC apps" error when I tried to open it in SpotLight). When I tried to open a copy of the readme.md file, it tried to open Xcode, which I haven't used in so long that I don't even have an active copy on my current laptop. So, I exported a copy of my edited Index.html file to my old laptop (where Xcode IS still installed). When I opened the file in Xcode, I saw the header info before the actual code. When I deleted it (and the end of line slashes), saved & exported it back to my current laptop, it now runs cleanly. I had also tried opening it with MS Word (apple version), but it likely just interpreted the RTF header info without displaying it. I always thought TextEdit was the simplest, "cleanest" file editor, but I learned something today - RTF is not the same as TXT. When I went into the settings of TextEdit, I saw that there IS an option to "Display HTML files as HTML code instead of formatted text". Had I used that option, I could have just edited & saved the code directly. Without that option checked, opening the file in TextEdit just displays the executed code. Alternatively, (just as a goof) when I added .TXT to the end (forcing it to be recognized as a simple text file), I saw the code you wrote and the RTF header junk, so I could have edited it that way, also. TXT files are "clean" & show all data within, but RTF files have extra header/formatting info that gets interpreted without being displayed. Ya learn somethin every day...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now