Find Duplicate Files - Windows - Powershell

[PenkethBoy 2066]

This is a simple one line script to highlight duplicate files - By NAME

How to use the script

Very briefly - I am assuming you know how to run a PS Script - if you dont Google it.

Run this script from the root of say your movies directory via the powershell console window (not the Powershell ISE if using PS5) - script is recursive so will find all movies

It may take a few seconds or minutes to look through all directories - depending on the number of files and speed of the pc.

File types searched for mkv, mp4, avi and ts - but if necessary you can add others

It will produce a txt file called ... drum roll ...Duplicates.txt - the file will be located in the same location as the script file and will get overwritten with each run.

If the file is empty after the script completes - then no duplicates found - otherwise it will contain the duplicate movies  (by name)

example output of a duplicate

with a duplicate "count" will be 2 or higher etc - "Group" will attempt to list the paths to each duplicate but if you have many files it will get truncated eventually.

If like me you have numerous extras - some are likely to be named the same - e.g. Behind the Scenes - so you will get false positives.

I have not filtered the search to exclude trailers, extras directories as you could have genuine duplicates there or misplaced files etc but it can easily be done.

Also the search is case insensitive

Duplicates.zip

Unzip the above and find your duplicates!

Have Fun

PenkethBoy 15/10/2020

Edited October 15, 2020 by PenkethBoy

[PenkethBoy 2066]

reserved

[PenkethBoy 2066]

reserved

[PenkethBoy 2066]

reserved

[rbjtech 4996]

Amazing for a 1 line script - and fast too.  Great work @PenkethBoy .. 

I'm now going to butcher it so it automatically scans all my media areas not just the directory it's run from .. 

[PenkethBoy 2066]

2 lines is your challenge?

[Jcheinaman 18]

This may be a silly question, but if I put this in my media directory, will it find movies inside of subfolders? All of my movies are organized into folders such as the example below.

 

Media/H/Halloween (2018) - 1080p/Halloween (2018) - 1080p.mkv

 

Basically, if I put the script in the Media folder, would it scan down into the other folders to find the duplicates?

Thanks!!

[PenkethBoy 2066]

thats what recursive means

[Jcheinaman 18]

On 2/9/2021 at 3:32 PM, PenkethBoy said:

thats what recursive means

Thanks

[HawkXP71 107]

The problem here is you only check by name.  If you have 2 files that are identical, there will not be a match. 

This is an app I wrote that uses MD5 on the files to compare

https://github.com/towel42-com/FindDupe

[Hispanico957 21]

In relation to finding duplicates, I wonder if there is a utility that can find similar files.

Let me explain, I have many movies both normal (1080) and with the same name in 4K, like for example:

The Mist (2007).mkv
The Mist (2007) 4K.mkv
The Others (2001).mkv
The Others (2001) 4K.mkv

How can I identify these films that have the same name but with a 4K ending ?

 

Thank

Hispa

[Luke 40063]

25 minutes ago, Hispanico957 said:

In relation to finding duplicates, I wonder if there is a utility that can find similar files.

Let me explain, I have many movies both normal (1080) and with the same name in 4K, like for example:

The Mist (2007).mkv
The Mist (2007) 4K.mkv
The Others (2001).mkv
The Others (2001) 4K.mkv

How can I identify these films that have the same name but with a 4K ending ?

 

Thank

Hispa

@rbjtechmay have some tips.