K-3 20 page review out

Smeggypants
Posted 20/11/2013 - 08:17 Link
screwdriver wrote:
And the purple defringeing gets applied globally too - interesting.
Purple Defringing is of no relevance to this discussion. Why you have randomly brought that up I have no idea.

However in the spirit of good discussion and attacking the argument and not the arguer I steer the debate back to the claim that RAW Files contain what you describe as "hexadecimal codes" denoting which pixels contain noise.

Chris A challenged you to "cite authoritative references supporting [your] claim that raw files contain metadata indicating where the noise is."

You haven't done this. Instead you've posted a screenshot of a hex editor's display of ACR meta data embedded in a RAW file of a picture of some snails. You've outlined some hex values in this screen shot and claimed:

screwdriver wrote:
the values surrounded by my rather crude outline are the pixels in the image to apply the de-noise to.
Of course they are not. They are hex values of ASCII Codes and spell out

". crs:LuminanceNoiseReductionDetail="

'LuminanceNoiseReductionDetail' is the standard nomenclature for ACR's noise reduction parameter.

Here's a Hex to ASCII converter to show it. http://www.dolcevie.com/js/converter.html

The same nomenclature is used in Lightroom's develop preset *.lrtemplate text files.


In laymen's terms ASCII codes = text, and ACR interprets this text to recognise a parameter and apply the value associated with it. The value in your case was 0x31:0x33 = 13

To claim that hex numbers that denote the text name ( in ASCII ) of an ACR parameter is the pixels to apply the noise reduction is showing a complete lack of understanding of both physics and how the NR process works.




.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Edited by Smeggypants: 20/11/2013 - 08:17
Crossed-up
Posted 20/11/2013 - 10:22 Link
Smeggypants wrote:
screwdriver wrote:
And the purple defringeing gets applied globally too - interesting.
Purple Defringing is of no relevance to this discussion. Why you have randomly brought that up I have no idea.

However in the spirit of good discussion and attacking the argument and not the arguer I steer the debate back to the claim that RAW Files contain what you describe as "hexadecimal codes" denoting which pixels contain noise.

Chris A challenged you to "cite authoritative references supporting [your] claim that raw files contain metadata indicating where the noise is."

You haven't done this. Instead you've posted a screenshot of a hex editor's display of ACR meta data embedded in a RAW file of a picture of some snails. You've outlined some hex values in this screen shot and claimed:

screwdriver wrote:
the values surrounded by my rather crude outline are the pixels in the image to apply the de-noise to.
Of course they are not. They are hex values of ASCII Codes and spell out

". crs:LuminanceNoiseReductionDetail="

'LuminanceNoiseReductionDetail' is the standard nomenclature for ACR's noise reduction parameter.

Here's a Hex to ASCII converter to show it. http://www.dolcevie.com/js/converter.html

The same nomenclature is used in Lightroom's develop preset *.lrtemplate text files.


In laymen's terms ASCII codes = text, and ACR interprets this text to recognise a parameter and apply the value associated with it. The value in your case was 0x31:0x33 = 13

To claim that hex numbers that denote the text name ( in ASCII ) of an ACR parameter is the pixels to apply the noise reduction is showing a complete lack of understanding of both physics and how the NR process works.




.
I think Chris mentioned purple fringing as this is something targeting specific pixels (unless purple saturation is just pulled down globally!), and so supports the claim that the same is happening with the de-noise parameter.

However, I totally agree that the highlighted bytes in the posted hex editor screenshot merely shows the hex values of the ASCii characters of the parameter name. There appears to be no reference to where the value is applied. I suppose it is conceivable that there is a memory map somewhere that contains references for the specific pixels that the parameter is applied to, but I can't see how that could be derived consitently.
My Images
K1 II|K3|DA10-17|DA*16-50|FA31|DA*50-135|DA*200|DA*300|Sigma 150-500
screwdriver
Posted 20/11/2013 - 11:53 Link
What your saying is true Smeggy, these are contained within ACR, but what your looking at here is the XMP file which is a look up table for the values of all the tools on opening the file. The preliminary start up values.

If all I wanted was to set the de-noise to 13 then all I would need here is two hexadecimal digits to set any value between 0 and 255 and start and end of data characters. The other hexadecimal digits are where to apply it, comprising the seed data for the algorithm within ACR for it's memory map flagging those pixels which are to have this setting applied.

There is no need to store all, in this case, 12Mp all it needs is the column (0 - 4000) and row (0 - 3000)in the memory map for the start pixel and then the algorithm within ACR will plot the others effected from that point and mark them to receive this data.

All Raw files open with the same tools in the same order, starting with brightness, then contrast and so on, once ACR has got these preliminary values in the XMP file it then moves onto the next tool then refers to this file for the parameters for that, the tool doesn't have to be called it's automatic in the program. There is no need to select the tool on opening as they always work in the same order, so the 'end of data' character in this XMP file makes ACR step through the program and move onto the next tool.

By this method it assembles the data for the XMP layer and creates it which, as I'm sure you know, sits above the DE-mosaiced file and adds the original values to those we have applied, we see the original data though this layer and modified by these settings, similar to how a colour profile works.

Any pixel can have all, none or some of the data applied from all, none or some of the tools within ACR.

It has to be stressed that this is just what happens on opening the Raw file within ACR, any further editing, such as altering the de-noise will alter the value within this XMP file, but it will still only effect those pixels tagged for that tool.

When you open the file into PS it flattens these two layers together and opens them as a PSD file, now, unless you make selections, any adjustments will be global.

The de-noise you refer to is under these conditions and then it's different.
Chris
dcweather
Posted 20/11/2013 - 13:22 Link
I suspect this has drifted way outside most people's expectation of "K-3 20 page review out" thread
michaelblue
Posted 20/11/2013 - 16:35 Link
screwdriver wrote:
What your saying is true Smeggy, these are contained within ACR, but what your looking at here is the XMP file which is a look up table for the values of all the tools on opening the file. The preliminary start up values.

If all I wanted was to set the de-noise to 13 then all I would need here is two hexadecimal digits to set any value between 0 and 255 and start and end of data characters. The other hexadecimal digits are where to apply it, comprising the seed data for the algorithm within ACR for it's memory map flagging those pixels which are to have this setting applied.

There is no need to store all, in this case, 12Mp all it needs is the column (0 - 4000) and row (0 - 3000)in the memory map for the start pixel and then the algorithm within ACR will plot the others effected from that point and mark them to receive this data.

All Raw files open with the same tools in the same order, starting with brightness, then contrast and so on, once ACR has got these preliminary values in the XMP file it then moves onto the next tool then refers to this file for the parameters for that, the tool doesn't have to be called it's automatic in the program. There is no need to select the tool on opening as they always work in the same order, so the 'end of data' character in this XMP file makes ACR step through the program and move onto the next tool.

By this method it assembles the data for the XMP layer and creates it which, as I'm sure you know, sits above the DE-mosaiced file and adds the original values to those we have applied, we see the original data though this layer and modified by these settings, similar to how a colour profile works.

Any pixel can have all, none or some of the data applied from all, none or some of the tools within ACR.

It has to be stressed that this is just what happens on opening the Raw file within ACR, any further editing, such as altering the de-noise will alter the value within this XMP file, but it will still only effect those pixels tagged for that tool.

When you open the file into PS it flattens these two layers together and opens them as a PSD file, now, unless you make selections, any adjustments will be global.

The de-noise you refer to is under these conditions and then it's different.
......and of course all this will make you a much better photographer
Regards,
Michael
ChrisA
Posted 20/11/2013 - 16:44 Link
screwdriver wrote:
but it will still only affect those pixels tagged for that tool.
Please. Cite just one reference from an obviously authoritative source substantiating this claim.

And explain why, if the pixels containing noise are identifiable, why noise reduction isn't a whole lot better.

As the saying goes, it ain't what people don't know that's the problem, it's what they do know, that ain't so.
.
Pentax K-3, DA18-135, DA35 F2.4, DA17-70, DA55-300, FA28-200, A50 F1.7, A100 F4 Macro, A400 F5.6, Sigma 10-20 EXDC, 50-500 F4.5-6.3 APO DG OS Samsung flash SEF-54PZF(x2)
.
Edited by ChrisA: 20/11/2013 - 16:49
Crossed-up
Posted 20/11/2013 - 17:11 Link
I think the issue here is not that the the XMP files applies the adjustment to the selected pixels, but how those pixels were selected or tagged in the first place within the RAW file. If they were selected by the user performing some sort of selective processing in ACR etc., then fair enough, but I still fail to see how the proportion of noise within a pixel value can be determined at the RAW capture stage
My Images
K1 II|K3|DA10-17|DA*16-50|FA31|DA*50-135|DA*200|DA*300|Sigma 150-500
Smeggypants
Posted 20/11/2013 - 18:23 Link
First you tell us that a camera can add metadata to a RAW file so that Lightroom can tell which pixels are affected by noise and by how much. Shown to be nonsense

Then you tell you know what PHP is, and you clearly don't

Then you post a screen shot of a Hex editor's output of RAW metadata, roughly outline the hex of ASCII codes for a parameter name ( LuminanceNoiseReductionDetail ) and tell us that's "data from the camera giving the location in the memory map of what the camera thinks are the effected pixels" which of course is nonsense

And now you've presented us with the following diatribe which I can only describe as obfuscated, technobabbled, grammatically disjointed waffling gobbledegook attempting to describe how ACR reads and applies the RAW file's metadata.

Comment Image


screwdriver wrote:
What your saying is true Smeggy, these are contained within ACR, but what your looking at here is the XMP file which is a look up table for the values of all the tools on opening the file. The preliminary start up values.

If all I wanted was to set the de-noise to 13 then all I would need here is two hexadecimal digits to set any value between 0 and 255 and start and end of data characters. The other hexadecimal digits are where to apply it, comprising the seed data for the algorithm within ACR for it's memory map flagging those pixels which are to have this setting applied.

There is no need to store all, in this case, 12Mp all it needs is the column (0 - 4000) and row (0 - 3000)in the memory map for the start pixel and then the algorithm within ACR will plot the others effected from that point and mark them to receive this data.

All Raw files open with the same tools in the same order, starting with brightness, then contrast and so on, once ACR has got these preliminary values in the XMP file it then moves onto the next tool then refers to this file for the parameters for that, the tool doesn't have to be called it's automatic in the program. There is no need to select the tool on opening as they always work in the same order, so the 'end of data' character in this XMP file makes ACR step through the program and move onto the next tool.

By this method it assembles the data for the XMP layer and creates it which, as I'm sure you know, sits above the DE-mosaiced file and adds the original values to those we have applied, we see the original data though this layer and modified by these settings, similar to how a colour profile works.

Any pixel can have all, none or some of the data applied from all, none or some of the tools within ACR.

It has to be stressed that this is just what happens on opening the Raw file within ACR, any further editing, such as altering the de-noise will alter the value within this XMP file, but it will still only effect those pixels tagged for that tool.

When you open the file into PS it flattens these two layers together and opens them as a PSD file, now, unless you make selections, any adjustments will be global.

The de-noise you refer to is under these conditions and then it's different.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Smeggypants
Posted 20/11/2013 - 18:24 Link
Crossed-up wrote:
I think the issue here is not that the the XMP files applies the adjustment to the selected pixels, but how those pixels were selected or tagged in the first place within the RAW file. If they were selected by the user performing some sort of selective processing in ACR etc., then fair enough, but I still fail to see how the proportion of noise within a pixel value can be determined at the RAW capture stage
It can't. If it could we'd have noise free images at all ISOs.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Crossed-up
Posted 20/11/2013 - 18:48 Link
Smeggypants wrote:
Crossed-up wrote:
I think the issue here is not that the the XMP files applies the adjustment to the selected pixels, but how those pixels were selected or tagged in the first place within the RAW file. If they were selected by the user performing some sort of selective processing in ACR etc., then fair enough, but I still fail to see how the proportion of noise within a pixel value can be determined at the RAW capture stage
It can't. If it could we'd have noise free images at all ISOs.
Exactly! I've spent years at work (ADC and power line signalling, not imaging data) trying to explain to people why removing unwanted noise from a data stream or signal is impossible without knowing what the original signal or noise was.
My Images
K1 II|K3|DA10-17|DA*16-50|FA31|DA*50-135|DA*200|DA*300|Sigma 150-500
pgweber
Posted 20/11/2013 - 18:59 Link
But what about if dark frame subtraction was applied, a la K10D?
i.e. when an exposure is repeated but without the shutter opening, so that no image is recorded on the sensor, just noise?

That would give a noise map.

Is it still available on the K5?

I never really used it on the K10.
Peter

Pentax K5
Pentax DA 18-55 Mk1, 50-200 (Samsung), 16-45, 55-300 Mk1, 35 f/2.4
Pentax MZ6 + FA28-90, FA50 f/1.4, M 50 f/1.7
Tamron 80-210mm & 28mm
ChrisA
Posted 20/11/2013 - 19:26 Link
pgweber wrote:
But what about if dark frame subtraction was applied, a la K10D?
i.e. when an exposure is repeated but without the shutter opening, so that no image is recorded on the sensor, just noise?
The noise isn't in the same place each time. Real noise is random.

There's fixed-pattern noise... where different pixels have different responses.

This is what you can get rid of by dark frame subraction.

But real noise you can't - you have to average it out over multiple exposures, where the actual signal (which is there in the same place every time) eventually predominates over the noise (which isn't).

Disclaimer: I know nothing about fixed pattern noise except what I just Googled.
.
Pentax K-3, DA18-135, DA35 F2.4, DA17-70, DA55-300, FA28-200, A50 F1.7, A100 F4 Macro, A400 F5.6, Sigma 10-20 EXDC, 50-500 F4.5-6.3 APO DG OS Samsung flash SEF-54PZF(x2)
.
Edited by ChrisA: 20/11/2013 - 19:27
Stuey
Posted 20/11/2013 - 19:54 Link
And so, is it a good camera :
K10D, K5 plus plenty of clueless enthusiasm.

My Flickr site link
Smeggypants
Posted 20/11/2013 - 20:01 Link
pgweber wrote:
But what about if dark frame subtraction was applied, a la K10D?
i.e. when an exposure is repeated but without the shutter opening, so that no image is recorded on the sensor, just noise?

That would give a noise map.

Is it still available on the K5?

I never really used it on the K10.
I didn't use it either as it was annoying to wait 30 seconds before you could use thecamera again after taking a 30 second night shot. I once made the mistake of doing a 40 minute bulb exposure and had to wait another 40 minutes - ARRRGHHHHH!!!

But anyway that feature is nothing to do with what's being claimed an challenged here.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
pgweber
Posted 21/11/2013 - 18:17 Link
Smeggypants wrote:
I didn't use it either as it was annoying to wait 30 seconds before you could use thecamera again after taking a 30 second night shot.
Same here.

Smeggypants wrote:
I once made the mistake of doing a 40 minute bulb exposure and had to wait another 40 minutes - ARRRGHHHHH!!!
But how did the shot turn out?
Peter

Pentax K5
Pentax DA 18-55 Mk1, 50-200 (Samsung), 16-45, 55-300 Mk1, 35 f/2.4
Pentax MZ6 + FA28-90, FA50 f/1.4, M 50 f/1.7
Tamron 80-210mm & 28mm

Add Comment

To leave a comment - Log in to Pentax User or create a new account.



Proudly supporting Pentax User

Samsung Logo Asahi Pentax Logo