PEF or DNG??

Smeggypants
Posted 12/01/2011 - 01:26 Link
amoringello wrote:
Quote:
I would very much doubt the data is downgraded. Any compression would be lossless.
I would never keep a 30mb DNG file AND it's more compressed 16Mbversion. point? none.
I don't mean because of compression but because of the conversion from RAW to DNG.
I cannot imagine any reason to keep both either... unless there was some modification and/or degradation that would warrant keeping the original data.

It just makes me wonder why that option is there in the first place.
Maybe just a hold-over from when it was needed to keep people happy who were afraid that DNG was going to be problematic? Perhaps it is an option like an appendix -- now something that just isn't necessary anymore and should be removed?
I thought DNG was a RAW file, just in adobe format rather than proprietary format ???
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Smeggypants
Posted 12/01/2011 - 01:27 Link
Yes it is ...

http://en.wikipedia.org/wiki/Digital_Negative_%28file_format%29
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
amoringello
Posted 12/01/2011 - 01:56 Link
Nice description. It is in the TIFF format -- which would normally raise a flag that the data is munged to some extent and thats what always bothered me, but it goes on to say there is an extension to the data to describe the sensor pixel layout. As long as the Bayer filter information is properly retained you can get better results with extracting detail and noise reduction than if it was merely a basic TIFF or other reformatted set of data.

That was always my main question and never found a clear description before.
I should have looked at Wikipedia... much simpler than trying to read the full DNG technical specifications.

Never really seen a difference in results from RAW and DNG, but good to know the data should be sufficiently intact and I can stop losing sleep at night over it.

Thanks for the link.
Smeggypants
Posted 14/01/2011 - 04:41 Link
Just had a thought. K-5 has more bits per pixel than a K-7, K20,K10

So we can't really extrapolate on a megapixel basis
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Mannesty
Posted 14/01/2011 - 18:36 Link
IN LR3.3 (and probably earlier versions) there is an option in the Library menu to Convert [selected] photo's to DNG.

In my very quick test, all previous modifications to .PEF's are retained and a 13Mb .PEF file from my K20D converted to an 8Mb .DNG file with no discernible change in IQ.

That would seem to be a significant saving on disk space. I'm now in the process of converting all my .PEF's from last year (1972 files @ 27.7Gb) to .DNG format.

As a precaution, I have of course backed up my .PEF's first

Where's the 'fingers crossed' smiley when you need one. Comment Image
(Thanks Smeggypants).
Peter E Smith - flickr Photostream
Edited by Mannesty: 14/01/2011 - 19:01
Smeggypants
Posted 14/01/2011 - 18:50 Link
Mannesty wrote:


Where's the 'fingers crossed' smiley when you need one.
Comment Image
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Mannesty
Posted 14/01/2011 - 23:06 Link
My machine having spent several hours converting nearly 2000 .PEF's to .DNG's, I've gained a whopping . . . 1.2Gb of space

I'm very surprised , my initial trial suggested I would have saved much more.

EDIT: It seems that Windows 7 is misreporting the sizes of the converted files. I just used the standalone converter to convert 10 .DNG files to .DNG in the hope that they would be compressed.
File sizes of the output files looked to be up to 25% smaller but when I select the 10 original files and select properties, the size of all 10 files amounts to 145Mb. When I performed the same exercise on the 10 apparently smaller files, their size on disk was the same, 145Mb.
Peter E Smith - flickr Photostream
Edited by Mannesty: 14/01/2011 - 23:21
Smeggypants
Posted 14/01/2011 - 23:11 Link
Crikey! - I'm about to get the Adobe DNG convertor and try converting DNGs to DNGs.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Smeggypants
Posted 15/01/2011 - 01:32 Link
Ok, installed Adobe DNG converter 6.3


From my new K-5 ...

25Mb PEF gets converted to a 19Mb DNG

33MB DNG gets converted to a 22MB DNG.

I think I'm gonna shoot PEF in future and convert to DNG to get the 19Mb.

I'm just concerned about how this affects processing though, given the compression overhead.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
mera
Posted 15/01/2011 - 02:12 Link
I think one difference between PEF and DNG is that in DNG the camera stores a colour matrix data to inform the RAW convertor how the colours in the RAW file should be interpreted to match the manufacturers intention. Thus a good RAW converter that was produced before a camera release can use this information for the "unknown" camera and still do a good job with the colours. This data might consume a bit of space compared to PEF.

With my Pentax K7 I shoot DNG which are typically 11 Mbytes in the camera. I then use Lightroom to import these into my computer and I get Lightroom to convert them to DNG again on import. The DNG file stored on the hard disk is normally compressed even further, in the example above it may occupy 10 Mbytes on the hard disk. The non-realtime processing algorithms in the PC can do a better job with the data than the camera can.

Note:
- With the Pentax K10D & K20D the DNG files are not compressed.
- With the Pentax K7 & F5 the DNG files are compressed.

The noticeable variability in the file sizes shows they are compressed.



Paul
Smeggypants
Posted 15/01/2011 - 03:08 Link
by using DNG to DNG conversion on tonight's K-5 shoot, I reduced 28.4Gb down to 20.8Gb. Worth doing!
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Edited by Smeggypants: 15/01/2011 - 03:08
Mannesty
Posted 15/01/2011 - 07:45 Link
Mannesty wrote:
It seems that Windows 7 is misreporting the sizes of the converted files. I just used the standalone converter to convert 10 .DNG files to .DNG in the hope that they would be compressed.
File sizes of the output files looked to be up to 25% smaller but when I select the 10 original files and select properties, the size of all 10 files amounts to 145Mb. When I performed the same exercise on the 10 apparently smaller files, their size on disk was the same, 145Mb.
My edit from above.

That's the kind of compression I was expecting, but it didn't happen. It looked as though it had happened, but on closer inspection, it hadn't.
Peter E Smith - flickr Photostream
pgweber
Posted 15/01/2011 - 09:49 Link
Smeggypants wrote:
by using DNG to DNG conversion on tonight's K-5 shoot, I reduced 28.4Gb down to 20.8Gb. Worth doing!
Crikey!

How many shots was that?
Peter

Pentax K5
Pentax DA 18-55 Mk1, 50-200 (Samsung), 16-45, 55-300 Mk1, 35 f/2.4
Pentax MZ6 + FA28-90, FA50 f/1.4, M 50 f/1.7
Tamron 80-210mm & 28mm
pgweber
Posted 15/01/2011 - 12:10 Link
I have just downloaded Adobe DNG Converter 6.3 and ran it on 33 DNG files shot in 2009 with my K10D (apart from this one shoot, I have always used PEFs due to the smaller file size).

For originals ranging from 16,853,617 to 17,126,478 (average 16,999,259), the converted files ranged from 7,015,640 to 10,185,090 (average 8,852,933).

The 'converted' files ranged from 41.3% to 59.7% of the size of the originals (average 52.1%).

I suppose the next thing to try is to convert the conversions. If there was another significant size reduction, then I might begin to suspect some lossy compression.

Regards,
Peter

Pentax K5
Pentax DA 18-55 Mk1, 50-200 (Samsung), 16-45, 55-300 Mk1, 35 f/2.4
Pentax MZ6 + FA28-90, FA50 f/1.4, M 50 f/1.7
Tamron 80-210mm & 28mm
Smeggypants
Posted 15/01/2011 - 18:01 Link
pgweber wrote:
Smeggypants wrote:
by using DNG to DNG conversion on tonight's K-5 shoot, I reduced 28.4Gb down to 20.8Gb. Worth doing!
Crikey!

How many shots was that?
Just over 1100.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283

Add Comment

To leave a comment - Log in to Pentax User or create a new account.



Proudly supporting Pentax User

Samsung Logo Asahi Pentax Logo