Archive Storage - RAW vs JPEG
Posted 09/08/2011 - 03:59
Link
Anvh wrote:
I understand that when you don't work with lightroom or other such software.
If you did everything in lightroom it only takes on click a button to make a JPG so the way i see it,it takes more effort to manage those JPGS then to create them when you need them.
No it doesn't Most of My RAWs have long since been archived to DVDRs or more recently HDDs and stored away somewhere.I understand that when you don't work with lightroom or other such software.
If you did everything in lightroom it only takes on click a button to make a JPG so the way i see it,it takes more effort to manage those JPGS then to create them when you need them.
If I worked your way, I would have to go and get the DVDR, or HDD, hook it up to the computer, find the RAW, copy it to the computer, load up the Catalogue in LR I use for RAWs, locate the RAW image and export it.
That's a pain in the arse compared to just having the image there already.
I'm just running a 'properties' in Wondows explorer of my total JPG collection. When it finsishes I'll let ya know how many pics and how mig it is .....
OK done.....
#images = 127,282
Size on disc = 34.7Gb
This is my collection of pics since 1985. ( I've still about 9,000 35mm negatives to scan from my 1980s snapping days, but that is a long term ongoing project which will probably take me another 5 to 10 years, as I fit it in an around current photography and commitments )
Those 130,000 pics are all keyworded and instantly available.
My workflow may seem complicated, but it's only complicated to describe with words. In practise it's very efficient.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Posted 09/08/2011 - 07:22
Link
Smeggypants wrote:
If I worked your way, I would have to go and get the DVDR, or HDD, hook it up to the computer, find the RAW, copy it to the computer, load up the Catalogue in LR I use for RAWs, locate the RAW image and export it.
If that is the case you don't work in my way If I worked your way, I would have to go and get the DVDR, or HDD, hook it up to the computer, find the RAW, copy it to the computer, load up the Catalogue in LR I use for RAWs, locate the RAW image and export it.
All the RAWs are directly available on the PC with me.
Posted 09/08/2011 - 16:34
Link
Anvh wrote:
All the RAWs are directly available on the PC with me.
I'd need several Terabytes of storage space to do that. Smeggypants wrote:
If I worked your way, I would have to go and get the DVDR, or HDD, hook it up to the computer, find the RAW, copy it to the computer, load up the Catalogue in LR I use for RAWs, locate the RAW image and export it.
If that is the case you don't work in my way If I worked your way, I would have to go and get the DVDR, or HDD, hook it up to the computer, find the RAW, copy it to the computer, load up the Catalogue in LR I use for RAWs, locate the RAW image and export it.
All the RAWs are directly available on the PC with me.
but even if I did have enough space, I'd still work the same way. the JPG Lightroom catalogue is much more efficient to manage as the files are much smaller. So I do all my leywording, rating and slideshow viewing on the JPG Catalogue.
Teh other way I work is that every image had had some editing done. For example if I take 300 snaps at a location where the scene is roughly the same, I will create a temporary develop present with a bit of enhancements and apply that to all 300 in one hit. Takes a minute. I then go and do further refinements to the good ones.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Posted 09/08/2011 - 19:08
Link
Don't forget that some RAW files can be further compressed by WinRar or Winzip. WinRar should be able to compress each RAW file into it's own RAR file as a batch operation. I keep meaning to do that with some of my older stuff. Later DNG's are usually heavily compressed already.
Half Man... Half Pentax ... Half Cucumber
Pentax K-1 + K-5 and some other stuff
Algi
Pentax K-1 + K-5 and some other stuff
Algi
Posted 09/08/2011 - 19:17
Link
we try to upgrade our gear on a three year rotation when possible.... so an old computer gets saved witht he old os and old drives that way we can open up legacy programs and files in the future..
for example our old emac.... when we retired it from photoshop duties, we cloned the old hard drive to a new drive then put the new drive away... the old computer got a bunch of games loaded onto it and handed down to our youngest... if I ever need to open a legacy file or program, all I gotta do is boot the old computer off the stored drive and everything is still there...
for example our old emac.... when we retired it from photoshop duties, we cloned the old hard drive to a new drive then put the new drive away... the old computer got a bunch of games loaded onto it and handed down to our youngest... if I ever need to open a legacy file or program, all I gotta do is boot the old computer off the stored drive and everything is still there...
Fired many shots. Didn't kill anything.
Posted 09/08/2011 - 19:34
Link
Smeggypants wrote:
I'd need several Terabytes of storage space to do that.
And now you don't need several TB of storage... i wonder where you left your photos?I'd need several Terabytes of storage space to do that.
You actually need more room now because of the JPGs or don't you keep the RAW files?
So you search with the JPG and then look for the matching RAW file?
Doesn't sound straight forward to me but we all have our own way of working.
Algernon wrote:
Later DNG's are usually heavily compressed already.
When I import the photos i let lightroom convert them to DNG even when the files from the camera are in DNG.Later DNG's are usually heavily compressed already.
Lightroom compress them quite a bit more then, tried with RAR once to compress them more but you gain very little then.
Posted 09/08/2011 - 22:04
Link
Anvh wrote:
You actually need more room now because of the JPGs or don't you keep the RAW files?
As said I keep all raw files, except obvious duds, and archived them onto TWO 2TB HDDs, which I then store somewhere safe.Smeggypants wrote:
I'd need several Terabytes of storage space to do that.
And now you don't need several TB of storage... i wonder where you left your photos?I'd need several Terabytes of storage space to do that.
You actually need more room now because of the JPGs or don't you keep the RAW files?
The exported JPGs, again as said, don't take up much room at all. only 34.7Gb amd that's for 127.282 of them.
Quote:
So you search with the JPG and then look for the matching RAW file?
Doesn't sound straight forward to me but we all have our own way of working.
I don't need to look fro the matching RAW file unless I need to re-edit something. And I keep the last few months worth of RAWs on the computer anyway, until I get low on room again, and need to buy another TWO 2 TB HDDs to back archive them onto.So you search with the JPG and then look for the matching RAW file?
Doesn't sound straight forward to me but we all have our own way of working.
btw - When I remove RAWs from the computer I leave the RAWs folder structure intact and the lightroom RAW catalogue still points to those locations. If in the event I need to re-edit something I simply load it back onto the computer and open Lightroom RAW catalgue and ready for editing. It actually hasn't happened yet.
Except the remaining 35negs from the 1980s I haven't a scanned yet, I have my total image collection on the computer and keyworded in lightroom ready for viewing at anytime. works great. I make extensive use of Smatart collections, so I can sit and enjoy a slideshow of any combination of keywords, or the best of a shoot, etc,etc
All works very efficiently and is ergonomically.
Quote:
Lightroom compress them quite a bit more then, tried with RAR once to compress them more but you gain very little then.
Yes I convert DNGs to DNG, as well as PEF to DNG, using Adobe DNG convertor. it reduces size in both cases
Algernon wrote:
Later DNG's are usually heavily compressed already.
When I import the photos i let lightroom convert them to DNG even when the files from the camera are in DNG.Later DNG's are usually heavily compressed already.
Lightroom compress them quite a bit more then, tried with RAR once to compress them more but you gain very little then.
[i]Bodies: 1x K-5IIs, 2x K-5, Sony TX-5, Nokia 808
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283
Posted 10/08/2011 - 11:01
Link
I shoot RAW and all my photos are currently backed up RAW onto a HDD, additionally I use Flickr as a back-up for the JPEGS. I am however shortly going to evaluate my archive and split into three. Family album type photos, which I will keep backed up as JPEGS. 'Decent' photos, which will be backed up on two HDD's as RAW and JPEGS and from these 'Select' photos, additionally backed up RAW offsite. 'Decent' photos will only be around 20% of the current archive. Any photos not in 'Decent' will be initially kept on the computer only as Jepgs and purged annually.
Posted 10/08/2011 - 11:35
Link
I also shoot RAW. I keep two copies of my library, one on my home laptop, one on my office desktop, and use sync software to synchronise them every night over a broadband connection. This can be quite slow (upload speeds approx 1GB / 4 hours can't wait for fibre to the home), but it's good for my peace of mind. If I suddenly add a large amount of photos to one library I can save some time by also importing it to the other library, and only syncing the changes resulting from PP.
Like Dr O I also separate family album photos. In fact I usually don't even import those to Lightroom, but to iPhoto instead, which is easier for my wife to also use, and has nice simple Facebook sharing.
Like Dr O I also separate family album photos. In fact I usually don't even import those to Lightroom, but to iPhoto instead, which is easier for my wife to also use, and has nice simple Facebook sharing.
Tim
AF - Pentax K5, Sigma 10-20/4-5.6, Tamron 17-50/2.8, Sigma 30/1.4, Sigma 70-200/2.8, Tamron 70-300/4-5.6
MF - Vivitar CF 28/2.8, Tamron AD2 90/2.5, MTO 1000/11
Stuff - Metz 58 AF1, Cactus v4, Nikon SB24, Raynox 150, Sigma 1.4x TC, Sigma 2x TC, Kenko 2x macro TC, Redsnapper 283 tripod, iMac 27”, Macbook Pro 17”, iPad, iPhone 3G
Flickr • Fluidr • PPG • Street • Portfolio site
Feel free to edit any of my posted photos! If I post a photo for critique, I want brutal honesty. If you don't like it, please say so and tell me why!
AF - Pentax K5, Sigma 10-20/4-5.6, Tamron 17-50/2.8, Sigma 30/1.4, Sigma 70-200/2.8, Tamron 70-300/4-5.6
MF - Vivitar CF 28/2.8, Tamron AD2 90/2.5, MTO 1000/11
Stuff - Metz 58 AF1, Cactus v4, Nikon SB24, Raynox 150, Sigma 1.4x TC, Sigma 2x TC, Kenko 2x macro TC, Redsnapper 283 tripod, iMac 27”, Macbook Pro 17”, iPad, iPhone 3G
Flickr • Fluidr • PPG • Street • Portfolio site
Feel free to edit any of my posted photos! If I post a photo for critique, I want brutal honesty. If you don't like it, please say so and tell me why!
Add Comment
To leave a comment - Log in to Pentax User or create a new account.



7671 posts
16 years
Smegland
but I then export every shot to a new folder as a 1400x1000 ( max size ) jpg with a bit of compression.
I'm just wandering though, i only convert them to JPG when i need to and after the JPGS are used (uploaded, send) i delete them again since i won't use them anymore and you can simply create new ones if you've to.
There's no more work involved. I let a 3,000 pic set export overnight. Sorted.
The next day I import them into my Exports LR Catalogue and keyword them. That probably takes 30 minutes. I then rate them progressively as 2*, 3* and 4*. I then pic a bunch out of the 4* ratings for a gallery to post on Smeggy's. usually about 130 pics or there abouts. From that gallery I'll pick say 20 to reduce size to 800x800 and post here and other forums.
Running TWO Lightroom catalogues, one for Negatives ( RAWs ) and one for Prints ( JPGs ) works well for me.
Lenses: Pentax DA 10-17mm ED(IF) Fish Eye, Pentax DA 14mm f/2.8, Sigma 17-70mm f/2.8, Pentax-A 28mm f/2.8, Sigma 30mm F1.4 EX DC, Pentax-A 50mm f/1.2, Pentax-A 50mm f/1.4, Pentax-FA 50mm f/1.4, Pentax-A 50mm f/1.7, Pentax DA* 50-135mm f/2.8, Sigma 135-400mm APO DG, and more ..
Flash: AF-540FGZ, Vivitar 283