Our Great Photo Processing Shootout!

A proper test would involve running the same image through different software and applying a determined set of amkounts for each parameter... i.e brightness, contrast, tone curve, saturation, NR, sharpening, etc

Given the actual numerical values for different software would, for many paramters, be different foreach amount of effect then it would be best to apply the amounts in one software package and then use the other packages to try and make it look the same.

When you've done all that you'll find the differences to be minimal. I would predict most noticeable differences would be in things like NR.


it would all be pixel peeping

For me, the important differences in software is the actual features and workflow, not the technical quality of the output image

I would rate technical quality of the output first before features & workflow, but yes, they both are the prime considerations.

Probably been said earlier. I just Wash and Go. Lightroom4.What can Photoshop do after that? Real question.

I Don't know what I am doing. Guessing

Yep, all valid things to compare. But there is now a wider range of 'definition' processes available in some newer software, not really traditional sharpening at all. These processes can dramatically boost an images depth and realism, used carefully.

I know PS offers many multi-step ways to apply types of sharpening (mostly involving masking, layers, filters (like high pass) and different blend modes), but other software has alternative (more simple mostly) means to achieve some of this - for many users the added simplicity itself will raise the quality of output they potentially could achieve.

For a RAW converter, aren't retrieval of image detail, minimisation of demosaicing artifacts, noise reduction and sharpening the crucial elements that can be scored with reasonable objectivity - i.e. they require a better/worse decision not a "Do I like it?" decision.

Aren't the application of adjustments to white balance, brightness, contrast, tone curves and so on all very subjective? If an experiment compared these, wouldn't we would again be comparing the human input/skill/creativity rather than the effect of the software itself?

Andrew

I'm still reading this thread with interest and I've now come to the inevitable conclusion that I'm crap at processing images, regardless of the software used!

Regards
David

Lets put it down to a glitch David! I know you have produced many good looking images using PS before. Frankly, I envy you - I'd rather be out and about in the lakes in the misty blue murk with my delicious Pentax gear shooting pics than stuck indoors on my laptop doing all this PP!

Yes, this is looking increasingly like the type of factors that a valid test would need to limit itself to. The other more creative inputs will be there as well, and will obviously influence peoples responses, but we should be able to restrict how much individuals factor in these opinions when making a final preference.

Just for record, I too believe you may well be right. I also think others may go along with this. What we have all been critical of is the 'testing' methodology.

Like comparing DSLRs, we can learn little if sample images do not compare the same parameters. A review that compared a terribly exposed, mis-focused image from a k5 with a perfect image from a *ist D, might conclude that the latter was the better camera. The test would be meaningless.

I for one am certainly not knocking what you are trying to do, and have a genuine (and vested interest) in the results. I wish to encourage you in your endeavour, but would like conclusions that I could have confidence in.

No, they are all objectively analysable operations. i.e if you add +0.5EV of brightness to an image than that can be objectively quantified. The software either adds 0.5EV of brightness when you tell it to or it doesn't and adds some other value. The other issue is whether it adds that 0.5EV of brightness without any unwanted artefacts.

The whole failure of the test that created this thread was not specifying a common set of operations to apply to the same original image but using different software and instead letting the user run wild on their own subjective choices for values.

You can apply "do I like it" subjectivity to everything

100% absolutely! I can agree wholeheartedly with this evaluation. It has been said that I have not been listening to what people have been saying, but I have I promise. I have learned (speaking for myself now only :wink that we need these definitions and specific limitations for a measurable comparison. An EV setting test might be taking it too far - artifacts generated by this would surely be picked up when evaluating the image as a whole.

But these are the sort of things that should be done in a laboratory. this is a photographers forum, populated with human beings (and the odd Royal), and we want to have fun as well! The human factor is critical in my view.

We should allow some creative input to flow through, or else there won't be any real love or nurturing of the image. But I believe that by asking people to factor out those more subjective areas we can enjoy the image, the process, and get results!

I have just spotted the LinuxPhoto website which seems to have carried out tests on a whole ruck of (old versions of) RAW converters. He also makes his RAW test images available for download.

Do you folks think that replicating what Jacek Góźdź did and published on LinuxPhoto, but using the various current versions of the RAW converters in use by PU forum members might be a sensible approach? At the very least, the RAW downloads provide an available and fixed starting point for members that also includes white/grey cards so that colour/brightness/contrast manipulations can be measured too.

Andrew

Well you can all shoot me down as you know I dont use RAW but I would be interested in seeing a JPEG comparison of photo editors uisng the AHEM appropriate methodology

Keep up the good work both of you btw I have just read through every comment on all nine pages!

but not voted..........

Thanks for these new ideas on how to put software to the test. Well, I started all this (maybe more accurately 'it was all my fault'!), so I think its a good time to round it off.

I think we can mostly agree that in order to get any conclusive result on all the compared aspects of a photo, we need a more scientific, clearly defined and controlled test - I think more than we can achieve here.

There's been quite a polarization of opinion - one stream saying that mostly you could replicate the looks and qualities of a shot on any of the main professional editing tools we have.

The other stream saying that no, there will be certain aspects of an images toning, definition and other technical aspects that could only be achieved in a specific program or combination of programs. (I myself clearly fall into this camp).

What we can all agree on I think, is that this forum is probably not going to be the place to definitively demonstrate either. Thats not to say we shouldn't try and work through images on different software and see what is possible. Matthew from the PU team gave me an idea that has been used on ephotozine, where RAW files are put up for members to try their hand and compare results - this would be fun and informative here I'm sure, so I will look at launching something like this, and I hope to see some of you there as well.

I can say (and I'm sure David Storm would echo) that I very much appreciate the intense and detailed discussions, debates and critiques, both about the images and of the ideas behind the experiment. I hope that there has been at least some new thoughts or information on PP that has been useful to everyone who got involved.

Finally, my thanks to David for bravely putting his cracking image into the lions den, and thanks to all who voted and made critical comments on the processings. And of course, thanks for all those votes....now how many so far, what? only 48? oh come on people!..........