Can BC compare files irrespective of filename (Find duplicates)

[steve_77]

Ok so I'm wondering if the current version of BC can be used to find duplicates - I have a whole folder of jpg files (10k+) - without getting into too much detail im trying to find a way to get rid of the duplicates. The catch is i know i have duplicates - but cant match them up to a master set of files (And i certainly dont want to rename them all again). But is there a way BC, Devart Codecompare, or some other tool or script can find duplicates based on file CRC, data content and or date stamp - while completely ignoring any filename differences?



ie.

Img123.jpg 2.4M 11/23/2012 =?= SomeRenamedFile.jpg 2.4M 11/23/2012



this is particularly usefull... (as would PDF compare tools) that BC3 doesn't seem to offer either...

[Aaron]

Hello,

Unfortunately, BC3 currently only supports aligning by file name between two different locations. We do not have the ability to align on other criteria (CRC, data content, timestamp, etc) or scan a single location for duplicate items. Adding support for this is on our Customer Wishlist.

BC3 does support opening and comparing the text of PDF files in the Text Compare. It does not support comparing the pictures within a PDF, however. Is that the scenario you are running into?

[steve_77]

the compare by CRC (or any other means) would be great - Me and the Mrs were going through our wedding album and - trying to sort and catalog all the digital pics the photographers sent us, as well all the additional sources (there is a ton of photos 6-8GB) We lost track where we left off and accidentally combined some files with those in a separate directory that we thought we already sorted but - they got moved to the wrong place - so now we're left with a mix of duplicate files in different directory and are trying to find a way to check for duplicates, and get things resorted - as some of the files were renamed - instead of just Img<x>.jpg - most tools we have researched dont have CRC comparison functionality, and those that do have to have the same filename to work.

[steve_77]

The thing that let me to here is a post a long while back namely :

https://www.scootersoftware.com/vbul...ead.php?t=5435

And figured perhaps this type of feature was incorporated in the new versions. Unlike some of the other utilities i came across at :
http://www.techsupportalert.com/best...e-detector.htm

I actually trust BC and was hoping that they had a way to help resolve my dilemma being that the aforementioned post was 3 years ago, but perhaps not too many new features / changes since then. Correct me if I'm wrong, but you guys are still actively developing the BC tools right?

[chrroe]

Hi Steve,

this is really a F.A.Q. ...

Using the search function with "duplicates" brings a lot of topics.

This one I'd like to point out: http://www.scootersoftware.com/vbull...ead.php?t=5435

Maybe this helps ...


Bye
Christoph

[Aaron]

We've had many new features since then, but unfortunately have not been able to present a solid duplicate file finding solution. You can always find a list of our current development on the Download page, in the Change Log:
http://www.scootersoftware.com/ChangeLog.txt

Duplicate File Finding is still definitely on our radar, but is a significant project in scale. As you've found, there are entire programs dedicated to just that task.

[bcdewul]

@Aaron, the duplicate file finder, I think it has been on the wishlist for quite a while. Frankly speaking, I think it should be a kind integrated feature of Beyond Compare.

Of course you may say that users may then go for any other tool, but we as BC3 enthousiasts, we like the feature in BC.

Anyway, we hope this request will shift to a higher place on the wishlist :-)
=

[Aaron]

It is certainly a feature we'd like to incorporate. My point was more that some development teams spend all of their resources making a program that only does duplicate finding. So there are programs that do only this task, but if implemented into Beyond Compare it would be one of many things we can do. BC3 has many other features that require time and maintenance; we have not had the spare resources to implement a duplicate finder solution, yet.

[obetz]

Duplicate checking should be in Beyond Compare because it's the basis for handling renames and moves.

Even if plain dupe checking might not be in the scope of a comparison program, synchronising directories after renames and moves definitely is!

Oliver

[johnmeyer]

Duplicate checking should be in Beyond Compare because it's the basis for handling renames and moves.

I am evaluating the software because of a huge recommendation in the Sony Vegas forum. The software is quite good, but for the reasons described in this thread, it won't do me much good, because I have hundreds of thousands of files that need to be compared, but many of them have minor renames, but are still identical. Also, like already mentioned, I sometimes accidentally create copies in other folders. Thus, to be useful, I need two features: the ability to flatten the file structure and do comparisons across folders; and the ability to find files that are identical, even when the file name has changed. Without these two features, the program is going to be of limited use.

I own Ace Logic's Ace Utilities, and also FileBoss, and these programs both do some of these things, but neither has Beyond Compare's excellent user interface. The FileBoss compare feature may do what I want, but it is harder to deal with.

[Chris]

The current version of Beyond Compare can flatten folder structure. Matching identical files with name changes is still on our wish list for a future version of BC.

To flatten folder structure in Beyond Compare 3, load your folders in the Folder Compare, then select "View > Ignore Folder Structure" from the menu.

[Narbat]

Finding exact duplicates is easy. What I want is a way to find files that are almost, but not quite, duplicates. For example, find two photos that have identical image content but different metadata. Or match a resized/cropped/rotated/color-corrected photo with the original. Something that could score a degree of similarity (with adjustable weighting for various attributes) between two files rather than just a boolean same/different evaluation would be killer.

(Yes, I know how difficult this would be and I don't actually expect to see it. But a guy can dream, can't he?)