Must Be This Way

Home Dehydration | First steps with a Digital SLR Camera | First Steps in the Pyrenees | Home   

Photo Project: Archiving and Workflow

Photo

John Hesp writes asking if I have any advice on cataloguing images. This is a big area and one which I have some problems with. Nevertheless I’ve developed a workflow and archive system that works for me, so let’s look at some of the principles.

Masses of Data

Storing digital images can soon become a nightmare. It is very easy to collect a large collection of images. The old mantra that no digital file ever exists until you have two — separate — copies, is very true when it comes to images. It is easy to loose them. The first set of images I took in the Pyrenees with my current camera have all been lost, luckily I had jpeg copies of some of them on Flickr. But some have been lost for good. I have no idea how this happened!

Not only are there lots of images but each one if a big file. I encourage people to shoot in RAW and them process to jpeg or TIFF when needed. But RAW files are huge. Each of my 6 mega pixel RAW files is between 9 and 12 megabites. As the megapixel count goes up, of course, the files get bigger.

So, real consideration needs to be given to managing files. You can easily get into the kind of mess that you can with over full, email inbox. Some of the same management tactics work with images. Merlin Mann’s 43 Folders blog is an inspirational place when looking to deal with email overkill and some of his techniques can be transferred to dealing with images.

So, where to begin.

Keeping Copies

You need to copy your images as soon as you get them into the computer. I put every set of images into a unique folder. I have an external hard disc just for images. Immediately I then archive the files to CD ROM or DVD using Roxio Toast (I use a Mac). I then convert (copy) the RAW files on the hard disc to Adobe DNG format.

The DNG format is a ‘RAW’ format that Adobe have developed as an attempt to produce a public domain standard. Some companies, such as Leica, now use this as a default but Nikon, Canon and others, stick to their propriety standards. Over time standards fail. Having both RAW and DNG (digital negative) files gives me a little piece of mind if one or the other becomes obsolete! The files on the hard disc are automatically archived on a dedicated archive hard disc using Apple’s Time Machine software.

So, immediately I have 6 copies of each file. A RAW and a DNG file sit on CD ROM, another set are ‘used’ on hard disc and a further copy of each is archived along with everything else on my computer. You may think this overkill, but even this system has failed!

So, we end up with a lot of images in a lot of places! That’s even before we start playing with them and saving them in jpeg, tiff or Photoshop formats! Fortunately, there are now specialist catalogue programs to help us.

Catalogue Programs

There are now a number of these, but be a little careful. There are image manipulation programs that have some cataloguing abilities and there are cataloguing programs that have some manipulation abilities. I find that you need one of each!

At the top of the tree is probably Apple’s Aperture program. This is not cheap but it is an amazing product. It is a lovely attempt to mirror the old chemical/film processes. It shares some processing features with Adobe Lightroom. Aperture is too pricey for me and my Mac really isn’t up to it, but when I upgrade the computer I will upgrade to Aperture. Incidentally, Lightroom is now bundled with Photoshop to make a kind of Aperture competitor. Most, if not all, Aperture users will still have a copy of Photoshop (and we’ll see why in later posts).

These are expensive programs. Fortunately there are cheaper options!

Expression Media

The cataloguing program for the most of us is Expression Media, which used to known as iView. This program is now owned by Microsoft and (certainly on the MAC) it is now bundled with fuller versions of Office. I can be bought separately though. Expression Media (EM) works for both PC and MAC. Be warned though this is probably more expensive if bought separately. If you upgrade your Office version you’ll probably have it — you may even have it already. In the UK EM costs £139 — not cheap but good value for money.

EM catalogues all kinds of digital media but in reality it is really a photographic archiver.

EM works through Catalogs. Each catalogue can keeps records of images and allows you to find them by date taken, file type and so on. The Catalogs also preserve EXIF information. Images can be tagged for location of anything of your choice. Catalogs can include various ’sets’ to which you can add images. EM is very effective and fast. EM’s tagging facilities are excellent.

However, you will need to think hard about a system for using EM (or even Aperture). Professional photographers have to be very thorough about indexing and tagging images but then again then often have staff to do this for them! I start off with good intentions. I want to tag images with all kinds of data, with the location of where it was taken and so on. But in all honesty it is too much hard work!

But before I describe my own workflow system it is worth looking at a few other EM features. EM has some pretty decent manipulation tools and is very effective at conversion to JPEG or TIFF. EM knows where your files are. If you need to move them don’t do this from the desktop. Do this from the EM catalog and you’ll find that all of your catalog links are altered – very useful.

So, this is my (simple) workflow system …

Files downloaded from camera to hard disc. The are placed in a simple folder structure, for example. Pictures/Snowdonia 03 08/

Files copies/converted to DNG format (before the compact card is reformatted).

Files copied to CD ROM or DVD. Discs indexed using Roxio Disc catalog maker (comes with Roxio burn products).

Hard disc files backup with Time Machine.

In Expression Media I then add the files to a catalog.

I might create a new Catalog for a specifc event, let’s say Pete and Anne’s Wedding or Pyrenees 08.

Sometimes I will simply add to another catalog. For example, I have a single Catalog for Shropshire and for Snowdonia. These are automatically tagged for date taken and so are easy to find.

As I work on the files I add any variants to the catalog folder. So, JPEGS or complicated Photoshop (PSD) file edits are edited to EM. It is easy to bring up a list of all PSD or JPG files within EM.

If I have time I might add some extra tags but increasingly I rely on the bare minimum. But EM tags can be searched using Spotlight or Google desktop, andnd this can be useful.

I don’t go as far as keeping a DVD copy off site, but maybe I should!

However, those images I upload to Flickr and kept on their servers for ever (or whatever that means). As I said earlier, I’ve had cause to be grateful for Flickr as it has saved my bacon on a few occasions!

So, there we are ..

That is my system. Expression Media was not cheap to buy initially but the upgrade costs have been affordable. I do find that a program of this effectiveness is worth its weight in gold when keeping a large image archive. There are other options. Personally, I could live with Aperture but would prefer EM to the new Photoshop/Lightroom system, but that’s just personal choice.

Your system needs to be able to help you find images quite quickly without the use of complicated tags. You could replicate this functionality in a bespoke data base, data capture tool, but it wouldn’t be as comprehensive and would probably be slower.

This sees me through. I wish I was more disciplined at tagging though ….

posted by andy on 10.02.08 @ 7:53 pm | 10 Comments

10 Comments so far
Leave a comment

This is a good site:

http://www.savemymemories.org/

By Neil on 10.02.08 9:19 pm

Thanks Andy, very useful. I’ve found that whereas my ability to deal with camera technicalities has improved over time, my ability to deal with my growing number of images has got worse. iView was the software my pro friend advised before I even knew I needed it, so it’s good to see it endorsed here too. I think this will be something I invest in.

Thanks again for all the articles, it’s a resource I’ll be revisiting.

John

By John Hesp on 10.02.08 9:31 pm

An extra thought – with long term archiving it’s worth considering not only the lifespan of the backup sources but also your own lifespan! It’s not much good putting your images on 300 year gold discs such as These if you are the only person aware of all that work or how to retrieve it. It’s also good to be ruthless with editing to keep the flow manageable.

By Des on 10.03.08 9:03 am

I use a Mac and use Aperture and love it! but I do not use Aperture to its fullest potential (nor the camera for that matter), but am getting better as time goes on. I also try and backup and store as much as possible and Aperture with its vault is at least another level of backup. Earlier ipods are great as back up devices.

Once again well done Andy.

By Roger on 10.03.08 5:51 pm

Ruthless editing is always good! Ruthless tagging and organisation would be good too! Maybe I’ll index some more during the winter nights!

By andy on 10.04.08 5:14 pm

Adobe Bridge is actually quite good for managing your shots and is free with Photoshop CS2 (I don’t know about later releases). It interfaces well with Camera Raw and Photoshop, allows you to use a hierarchical directory structure to store your shots, allows batch processing and easy viewing/editing of metadata including keywords, grading of shots and searching. It may not have all the facilities of eg. Aperture and Lightroom but if you’ve already got CS2 you don’t really need to shell out for another package.

By Bob Martin on 10.05.08 9:48 am

I don’t think Bridge copes well with big collection. It is quite slow and nowhere near as powerful as a dedicated program such as Expression Media.

By andy on 10.05.08 4:58 pm

I downloaded the trial version of Expression Media 2 yesterday after reading your post and will be exploring it. Perhaps you could give me some pointers as to what sort of things I will be able to do with it that I couldn’t do with Bridge.

I’ve never found Bridge to be slow. Perhaps it depends upon what sort of folder structure you use to store your pics. I take your point that the size of the collection could affect the speed of operation but even a big collection can be broken down more efficiently using a hierarchical structure.

By Bob Martin on 10.05.08 8:11 pm

Having CS2 and Bridge I am trying to avoid shelling out on additional SW.
I wonder whether upgrading RAM would speed it up given the huge amount of data transfer required with RAW files.
I am upgrading mine to 2gb this week(costs less than 30gbp). Will report back.

By Fred on 10.05.08 9:05 pm

Fred/Bob,

As I suggested in the text,workflow is a very personal thing.

I do use Bridge quite regularly and sometimes find it useful. It is not IMHE ss powerful, flexible or a fast as a dedicated program such as Expression Media.

But all of these things come at a price. The most important thing is thta you have a workflow system that works for you. This must involve archiving religiously !!!

By Andy on 10.06.08 10:20 am

RSS feed for comments on this post.


Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

(required)

(required)




RSS Newsfeed

Twitter: @mustbethisway
Twitter (all other stuff): @Andrew_Howell

Must Be This Way Twitter Twibe #MBTW

The Colin Ibbotson Pages

Guide to Home Hydration

FIRST STEPS IN THE PYRENEES SERIES

Pyrenees Trail Journal 2008 — Cauterets, Ordessa, Néovielle

Cairngorms by Sleeper, mini-break report

First steps with a digital SLR camera

TGO 2006 TRAIL JOURNAL
TGO 2007 TRAIL JOURNAL
TGO 2009 TRAIL JOURNAL
NEW—TGO 2010 Trail Journal

TGO Planning Guide
The Going Lighter Guide



Lightweight Kit Lists

Contact by email

Categories

Archives

Search

Check Out ...

21 Hikes, 1 Ride/9 Peaks — Peace One Day
A Little Bit About Not A Lot
AktoMan
Alan Sloman’s Big Walk
Alastair Humphreys — Adventurer
Ali and Lay’s Mountaineering Blog
Backpackinglight.co.uk
Batish’s Blog (Hiking in Japan)
Beating the Bounds – Mark Richards’ Blog
Ben Collins — Cross Spain Walk – NEW
Cameron McNeish – ‘The Godfather’
Chris Townsend
Colin Griffiths — Colin’s Biking Bits
Colin Griffiths — Croft Hill Record
Colin Griffiths — Rich Gift of Lins
Collected Musings of a Hill Wanderer — John Hennesy
Cumbria Fell Raven
Daryl May’s Hike Through Britain
Dave Wood is Red Yeti
Dawn’s Outdoor Blog
Doodlecat: an alternative look at the great outdoors
ebothy
Fat Git Walking
Footprints Across Scotland — Paul Sammonds
Gayle E. Bird (and MIke)
Gyrovagus
Hard Light – Steve Walton
Hendrik's Hiking in Finland
Hennessy Blog
Hike Wales
Hike-Lite
I Would Rather Be Walking
I’m So Dave – LEJOG
James Boulter – Backpacking Bongos
John Hee’s Blog
John Manning
Judy Armstrong’s Alpine Challenge
London Backpacker
Lone Walker
Mark Alvarez
Martin Banfield – Postcard from Timperley
Mike Pitt
NEW — Laura LIddell
NEW — Louise's Big Adventure
NEW — Minimal Gear
NEW! — Self Powered!
Nielsen Brown
Northern Pies (Mike Knipe ate them …)
Peewiglet
Peewiglet’s Blog
Petesy’s Blog
Phil Turner – lightweightoutdoors.com/
Practical Backpacking Forum
Robin Evans – Blogpackinglight
Roman’s Lighthiker’s World
Ron’s Walking Fort Bragg Blog
Ryan Jordan
Section HIker — Phil Werner
Shed Dweller — John's Shed Dwelling Moments!
Solitary Walker
Steven Horner
Summit and Valley
Team io – Super ultralight gear in the UK
The Bearable Lightness, Gustav Boström
The Big Walk
The One and Only ‘Bearded Git’
The Roaming Dials
Three Peaks – Africa Expedition
Thunder in the Night — Joe Newton in Norway
Tommy Kelly
Two-Heel Drive
Walking with Paul Williams
Webtogs
Weird Darrren’s Whitespider1006 Blog

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.0 UK: England & Wales License.