The DAM Forum
Welcome, Guest. Please login or register.
October 24, 2014, 11:45:15 AM

Login with username, password and session length
Search:     Advanced search
28019 Posts in 5139 Topics by 2910 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Loss and Recovery
| | |-+  strategies for hunting and replaceing many corrupt files
« previous next »
Pages: [1] Print
Author Topic: strategies for hunting and replaceing many corrupt files  (Read 4724 times)
Mathew Farrell
Newbie
*
Posts: 22


View Profile WWW Email
« on: January 24, 2012, 11:49:42 PM »

Recently I've been discovering more and more corrupt images surfacing in my library. The most likely culprit is the external 2.5" hard disk or it's enclosure that I'm using as my key library disc.

I'm using Lightroom as a one stop shop for DAM stuff presently. Lately the catalogue and all processing has been run off one machine, a macbook pro. the LR catalogue is stored locally, backed up the external (lightroom's own cat' backup feature). Images are ingested from CF cards straight to the external drive (input via LR, convert to DNG on input). Using LR's input dialogue, a second copy (not DNG converted, not renamed) is saved to the internal system drive on the MBP.

I've recently moved interstate and don't have my archive backups on hand. Images ingested in the last 6 months are still on the MBP (the RAW, non-renamed files), although corrupt images seem to be scattered through the catalogue, more frequently in more recent files, though. Older images are TIFFs from my film scans. Some of these seem affected too.

I've cloned and quarantined the suspect drive until I can put in a box to run SMART data tests on it. Mac's Disk Utility can't fault it (I am connecting it via Firewire though - no eSATA on 13" MBP).

Does anyone have any good strategies for identifying and swapping out the corrupt files? I've tried to run the lot through DNG Convertor. This would be a great solution for a few images, but there are probably over 1000 corrupted, plus unknown TIFF casualties.
At present, I've been deleting corrupt files as they are found, then using LR's Import dialogue to find the original RAW files on my MBP backup. They then need to be renamed and reDAMed manually. This isn't a great solution for proving out the whole library though. When my backup drives arrive I'll investigate the library clone (now a couple of months old), and if that checks out, bring that up to speed with recent images and metadata, turning that into the main library drive (probably cloned to a new drive for various reasons). I'm bracing myself for things being more tricky than this, however.

Any thoughts or stories welcome.
Mat
Logged

Mathew Farrell
Flowstate Photography
photography.flowstate.com.au
Mathew Farrell
Newbie
*
Posts: 22


View Profile WWW Email
« Reply #1 on: February 04, 2012, 12:27:37 AM »

Update, I've just discovered a clean way to ID the damaged files:

I just figured on a variation of this method - select all images in your catalogue and export them to the smallest possible JPG files. I export to 1px x 1px, lowest quality, etc. That way the resultant files end up something like 8kb rather than megabytes. It takes a similar amount of time to run, but doesn't choke up a load of disk space in the meantime.

The key benefit of doing this through LR rather than DNG converter is that LR keeps a record of files that failed to export. Add a keyword to each of these files (such as Corrupt), or drop them all into a new set/collection. DNG convertor simply gives you a list, rather than helping you group or tag the files in question.
Another benefit of trying to convert everything. For example, as stated above, DNG convertor can simply ignore a file that is really mashed. The LR catalogue, by contrast, expects this to be an image file and reports it as an error.

I just posted this elsewhere in response to another's question

A drawback is that it list less critical problems (for me at least) too, such as metadata mismatches, etc.
Logged

Mathew Farrell
Flowstate Photography
photography.flowstate.com.au
Mathew Farrell
Newbie
*
Posts: 22


View Profile WWW Email
« Reply #2 on: May 21, 2012, 12:34:16 AM »

A happy update for this issue - I finally got my backup disks back on premises and fixed my library.

I used Lightroom to flag all the problem images as described as above. Ideally I would have then used LR to delete those particular files but not the catalogue entries. No such option exists (hardly surprising).
Rather, I backed up the catalogue, got LR to delete the files and entries, then used Carbon Copy Cloner to replace the now missing files with the (hopefully) good copies on my backup drive. Then replace the 'current' catalogue file with the backup I just created - the images are all there, so are catalogue entries and all metadata (by virtue of it being saved to the catalogue rather than the files).

The only hitch with this operation is that there are a bunch of outtakes and trash files that I had deleted from my catalogue which are now back on my drive. I used LR to hunt for and import any images that were now on the library hard disk but not in the catalogue. I will go through these individually, and re-delete garbage, etc. It's a good opportunity for a double-check that nothing valuable has been deleted in the past... That's the optimist in me talking, mind you. In reality it's another job getting in the way of actually taking photos.
Logged

Mathew Farrell
Flowstate Photography
photography.flowstate.com.au
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!