Software to detect and delete duplicate images

HI
Does anyone know of any reasonably good software that I could use to start reorganising my pictures as I have reached the stage that I better start it all over again. Like many I have ended with lots of duplicates scattered on my C drive and I had started backing them up on my external one so its all a mess now.

To make matters worse I have a laptop also and wife loads pics in there from her digicam that I have passed on to her. The main pictures which are of common interest and that I find more difficult to organise are the family and the little ones specially, as they are loads and loads of them and now quite a few duplicates.

Now that I have upgraded to DSLR and getting to grips with it I definitely have started separating my lot but its the old ones that I want to streamline and start making some sense with them.

Any suggestions or advice please?
Much appreciated

Amin

Amin,
I make heavy use of Scooter Software's Beyond Compare.

A lifesaver for synchronising multiple drives
Matt

For a byte by byte comparison to find dupes in whatever directories you throw at it Check Identical Files by http://www.identicalfiles.com
Costs 29USD

shim

Oh super! I've been meanig to ask this sort of question for a while.

Also, is there any software for cataloging images? By that I mean, if ever I get around to adding tags, is there any software I can use to search by the tags. ( Please don't say windows already does this )

An expensive version which I use is Adobe Lightroom (academic discount helped!) but I know some other users recommend other cheaper options.

Best wishes, Kris.

Amin,

Brilliant question! Why didn't I think of it. I have the same problem.


MattMatic,

You can always be relied on to come up with the answer!



Shim, thanks as well.

Now I know what I'm going to do for the weekend!!

hehehe this is the sort of question I have been wanting to ask for quite a while but was afraid of spelling it out in case I would be told off ha (just joking) thought it would expose my utter ignorance about digital photography world, but I see that I wasn't the only one searching for something like this. Oh my, this is such a relief hehehe, so I am not that dumb after all, couldn't be if I am in such great company. So many of you also looking for this huh? . Nice one.

And Matt, you are marvelous as always, and thanks for everyone to chip in with suggestions. I am off to check the applications and clean my folders. Its starting to get to my nerves when I open the Vista image viewer with so many duplicates and some that I should just delete it. The rate at which the images are increasing now since I got my K20D has gone up the roof so I got to keep a tab on them and start checking and rechecking and then delete some.

Thanks again everyone

Amin

Hi Amin,

I use Picasa. Simple to use, has a few basic tools included, that don't do a bad job if you haven't gotten around to getting a decent image program such as PS or GIMP, and free. I tend to create a new folder for each trip out, named something along the lines of "Lake District 15072008", so I can keep track of my images easily. In Picasa, all your folders that contain images (it automatically scans your drives for you, and identifies any image extensions such as *.jpg, *.PEF (that's your raw files) etc. and lists the folder for you). That's down the left hand side. The main panel thumbnails all your images.. including *.PEF. I find that particularly handy, as Windows doesn't 'see' raw files unless you use a plugin, which I didn't get to work. You can add captions to each image, and tags.. although I don't use tags..

It will also find duplicates, go to Tools | Experimental, and aparently it is stable.. again I don't use that.

It's free, and any changes made to a photo can be undone.. it holds a metafile with your changes in the same folder as the image. I always keep an original and work on a copy though, and I'd advise the same to you. Belt and braces..

You can also use it to upload to a blog, to your own web gallery, and if you want to make a new button for it, you can open the image you want straight into which ever editing program you want.

Like most here, I have 10s of thousands of images, and can find them when I want them.

More info here.. http://garmahis.com/software/picasa-3-new-features/

Hope that helps,

Cheers

Liam

Liam

Thank you so much for your input on this. I really didn't know picasa had these extra features up its sleeve. I did play with it once long time ago and didn't give much thought to it because at that time I was looking for a more robust editor, which subsequently I managed to get hold of GIMP and am quite happy using it with my limited capabilities with these editing applications. In fact I am extremely happy with it.

Since then I never went back to picasa to explore extra features but I recall finding it very user friendly and the fact that it sorted out the pictures in the way it did the day I ran it captured my attention, but let it go because I was looking for an editor.

I will go back to it and start exploring further. My main problems reside in two aspects. First although the bulk of my pictures are all inside MY PICTURES, but I definitely have some scattered in other folders that get created occasionally just as temporary but then I never go back to it to empty and delete it, thinking that I would revert to them but never happens. I open sometimes folders when i am working on a webalbum creation for example, which I do often using jalbum and fotoplayer, so ideally I want a duplicate finder that would search my main drive rather than make a comparision between two folders, which seems like most of these applications are good at doing. I have briefly downloaded the ones suggested above but unless I am missing something they are good at comparing and then finding duplicates.

I do recall sometime ago when I had decided to glance around I did come across a couple which were mainly scanning the C drive and coming up with duplicate files, and mainly images, which then it opened in two windows to ascertain if one should be ticked off for deletion. So I am sure there are such applications around, must be a matter of searching then.

But as you say, if picasa does that job then thats a huge bonus, because as I said earlier I really liked the interface and the layout and the look of it. So off I am to download it and give it a try.

Thanks again

Amin

Liam

You mention how you create your folders, by the trips you make, so I am just thinking about the way I have been doing and see if I get some input or advice in improving the system.

Mainly my pictures are family and kids ones, and since I found that windows was giving me an option to upload my pics from the card by creating a new folder named with the date I have ended up with a huge number of folders based on date.

Whilst that helps me when I go looking for a particular picture because I know more or less the date when it was taken, and also the Vista browser opens up as an option based on the dates of the pictures, and then I can select and open the folder, I still think this is not the most ideal way of cataloguing the pictures and putting them in folders named by dates.

Consequence of this is sometimes I have very few pics taken over several dates and when I upload into the drive I end up with so many folders and many of them with just very very few pics.

Its ok in a way because they are all split by dates, but I am open to suggestions if there is a better way of creating folders or naming them in a way that might be more suitable. What are your experiences regarding this aspect? I presume picasa might give me some other options as well, so it would be interesting to know what other methods others are using.

Thanks very much for all your inputs.

Amin

My way of organising is basically by date:
F:/IMAGES/YEAR / DATE - DESCRIPTION

So, I'll have:
F:/Images/2009/2009-01-06 Test images/

Underneath that I'll have "/Silkypix" for TIFF & JPG outputs, "/Processed" for Photoshopped work, and "/Processed/Web" for web output.

Like Liam, I use Google's Picasa to organise and locate images. Unbelievably quick to find images and works a treat
(My wife likes using it for processing her *ist-DS RAW files too - nice and easy to use!).

I've used this organisation structure for years and have everything under control... except having to upgrade hard drives every couple of years (Currently on 750Mb LaCie Quadra).

Matt

Matt

true, for the past couple of hours I have been playing with picasa 3 and its unbelievably user friendly and nice to work with, even from the organisation point of view. Still shufling with it when I saw your comment pop in.

Also seems like the date structure of image folders is good one then, mine after all its more or less the same way, but the way you have done is you have opened the root folder for the year and then subfolders within them. I have named straight all the folders by dates so they are all scattered into the MY PICTURES folder. I think I will be able to streamline it better if I divide them by year also and then move the subfolders into the respective years.

Beauty of picasa is it finds and subdivides the folders by year even if they havent been opened and saved by year, hence the neat and simple aspect and easy to locate what we want. Vista does something like this but seems picasa does it with more flair and refinement. I like it, and I am sticking with it

Thanks again

Amin

Try looking into imatch, it has the ability to search for duplicate images, or visually similar images, or alternatively via the exif data such as original image creation date.

http://www.photools.com/products.php

I have had this on trial and am impressed, has a v.helpful user group too.

You can do no better than IDImager which, IMO, is the best image management software available, it's extremely powerful (which means it can be a bit daunting at first), is very affordable and has one of the finest support forums of any software that I know of. It integrates and works with just about every other program and all the major on-line image hosting services. Fully XMP compliant (bi-directional) and has true versioning support.

Hugely recommended

IDImager - link
Support forum - link
Comparative reviews (2) -
link
link

Testimonial from a Pentax user (not me!)
link

Windows XP & Vista (MAC via Parallels

I have used the Personal edition of Idimager and I would recommend it.