In our office workflow, I shoot RAW (converted to DNG on upload), which is all dropped in one folder.
Ideally I do a pile of keywording before even opening ACR.
ACR editing is done, and usable files are saved out as PSD files in a different directory.

The directory of PSD files is effectively available for other users to cherry pick images from. The DNG folder is a combination of our archive and a 'second resort' for savy users to hunt for an image.
Often I find other keywords that should have been added to images, or I got excited and created PSDs before keywording at all.

Does anyone know of a way to synchronise those keywords between folders?
Further info: The files bear the same name (excepting the extension), and the PSDs are always created through Bridge>ACR>Photoshop, so potentially carry a metadata identifier. There are many files in the DNG folder that never make it to the PSD folder.

Cheers all,
Mat

A happy update for this issue - I finally got my backup disks back on premises and fixed my library.

I used Lightroom to flag all the problem images as described as above. Ideally I would have then used LR to delete those particular files but not the catalogue entries. No such option exists (hardly surprising).
Rather, I backed up the catalogue, got LR to delete the files and entries, then used Carbon Copy Cloner to replace the now missing files with the (hopefully) good copies on my backup drive. Then replace the 'current' catalogue file with the backup I just created - the images are all there, so are catalogue entries and all metadata (by virtue of it being saved to the catalogue rather than the files).

The only hitch with this operation is that there are a bunch of outtakes and trash files that I had deleted from my catalogue which are now back on my drive. I used LR to hunt for and import any images that were now on the library hard disk but not in the catalogue. I will go through these individually, and re-delete garbage, etc. It's a good opportunity for a double-check that nothing valuable has been deleted in the past... That's the optimist in me talking, mind you. In reality it's another job getting in the way of actually taking photos.

Lightroom has definitely reduced the necessity of photoshop for many images. ACR would do the same, but I don't think you can buy or get it without Photoshop anyway.

The way my workflow goes, and probably many folks now is such that most images get some form of editing in LR or ACR, then only open specific files in photoshop for special/deeper work such as stitching/HDR (relevant for you perhaps), etching, spotting, etc.

If I were strapped for cash, I'd get LR rather than PS, though both is a blessing.

Hi Jonathan.

Cheers for the thoughts, and I'm glad to hear you're back in action.

I started a fresh post on this: http://thedambook.com/smf/index.php?topic=5227.0 outlining the details. I'm starting to suspect something similar to you, as I did migrate files a short while back without verifying (similarly now using CCC). It is conceivable that I'm only now finding problems that have been there since then. There are still too many frustrating unknowns at the moment. It's a frustrating position as any testing or experimenting, as well as taking time away from actually making pictures, also means walking further down the road with an untrustworthy system.

The chaps at work are testing the suspect drive in a machine so we can read SMART data (unavailable to me through the firewire external enclosure). It mounts and works 'fine', but they weren't able to get it to even show up in Disk Warrior... Too early to read anything into this though.

Update, I've just discovered a clean way to ID the damaged files:

I just figured on a variation of this method - select all images in your catalogue and export them to the smallest possible JPG files. I export to 1px x 1px, lowest quality, etc. That way the resultant files end up something like 8kb rather than megabytes. It takes a similar amount of time to run, but doesn't choke up a load of disk space in the meantime.

The key benefit of doing this through LR rather than DNG converter is that LR keeps a record of files that failed to export. Add a keyword to each of these files (such as Corrupt), or drop them all into a new set/collection. DNG convertor simply gives you a list, rather than helping you group or tag the files in question.
Another benefit of trying to convert everything. For example, as stated above, DNG convertor can simply ignore a file that is really mashed. The LR catalogue, by contrast, expects this to be an image file and reports it as an error.

I just posted this elsewhere in response to another's question

A drawback is that it list less critical problems (for me at least) too, such as metadata mismatches, etc.

I just figured on a variation of this method - select all images in your catalogue and export them to the smallest possible JPG files. I export to 1px x 1px, lowest quality, etc. That way the resultant files end up something like 8kb rather than megabytes. It takes a similar amount of time to run, but doesn't choke up a load of disk space in the meantime.

The key benefit of doing this through LR rather than DNG converter is that LR keeps a record of files that failed to export. Add a keyword to each of these files (such as Corrupt), or drop them all into a new set/collection. DNG convertor simply gives you a list, rather than helping you group or tag the files in question.
Another benefit of trying to convert everything. For example, as stated above, DNG convertor can simply ignore a file that is really mashed. The LR catalogue, by contrast, expects this to be an image file and reports it as an error.

That old chestnut!

It doesn't seem like we're due to see the end of those gremlins just yet.

One of the really nice things about Mac these days is they seem to have the edge on compatibility (not expecting a fire-fight here folks...) Are there any such considerations with Blu-ray drives?
Mat

Ryan, I don't understand the implications/problems of leaving drives on the shelf. I would have intrinsically assumed the opposite - that a constantly powered drive is more likely to blow a gasket. Could you explain for me (or point me another thread)?

Also a few comments here about not completely trusting optical. I understand the 'completely' part, but what are people's big fears? Is it mechanical damage (eg, scratches to the top)?

Cheers,
Mat

Hi Joe. A pleasure.

One thing I forgot to note, there's definitely a risk in relying on both of those drives in the one machine (theft, lightning strike, virus, etc). That risk is reduced as soon as you have a non-live and remote copy of your library - ie, external drive/Dropbox type account/DVD backup.

Both in the one machine is fine, but do make sure you've got a copy elsewhere too. Then build defense higher from there.
Mat

Recently I've been discovering more and more corrupt images surfacing in my library. The most likely culprit is the external 2.5" hard disk or it's enclosure that I'm using as my key library disc.

I'm using Lightroom as a one stop shop for DAM stuff presently. Lately the catalogue and all processing has been run off one machine, a macbook pro. the LR catalogue is stored locally, backed up the external (lightroom's own cat' backup feature). Images are ingested from CF cards straight to the external drive (input via LR, convert to DNG on input). Using LR's input dialogue, a second copy (not DNG converted, not renamed) is saved to the internal system drive on the MBP.

I've recently moved interstate and don't have my archive backups on hand. Images ingested in the last 6 months are still on the MBP (the RAW, non-renamed files), although corrupt images seem to be scattered through the catalogue, more frequently in more recent files, though. Older images are TIFFs from my film scans. Some of these seem affected too.

I've cloned and quarantined the suspect drive until I can put in a box to run SMART data tests on it. Mac's Disk Utility can't fault it (I am connecting it via Firewire though - no eSATA on 13" MBP).

Does anyone have any good strategies for identifying and swapping out the corrupt files? I've tried to run the lot through DNG Convertor. This would be a great solution for a few images, but there are probably over 1000 corrupted, plus unknown TIFF casualties.
At present, I've been deleting corrupt files as they are found, then using LR's Import dialogue to find the original RAW files on my MBP backup. They then need to be renamed and reDAMed manually. This isn't a great solution for proving out the whole library though. When my backup drives arrive I'll investigate the library clone (now a couple of months old), and if that checks out, bring that up to speed with recent images and metadata, turning that into the main library drive (probably cloned to a new drive for various reasons). I'm bracing myself for things being more tricky than this, however.

Any thoughts or stories welcome.
Mat

G'day Joe

I'm not the best qualified to answer your questions, but in short, the bigger the better, and it sounds like using one of your internal drives is a great way to go. As a running backup, an external drive of similar volume would be handy, or use the other internal drive (partitioned or not, as per your taste), with your archival backup to DVD or similar.

Whether you use the other internal drive or an external as a backup, they don't have to be the same size as the original library drive. It helps, as they fill up at the same rate, etc, but it's not important, unless you're using software which doesn't like synchronising different drives or some such.

If you don't already have it, get Peter's book (either edition really), which will answer a lot of this. Good luck, and do your best to do things right from day one!

Mat

Jonathan, did you ever diagnose the culprit? I'm experiencing a very similar problem.

I can't verify yet, but it sure seems like once-fine files are becoming damaged in recent days. My big frustration is that I've just moved interstate, and backup HDDs and DVDs are still weeks away.

Mat

Thanks John.

I presume that even with your plug-in, it's still a pretty manually intensive job?

Here's another one for you John:

A way to search out corrupted images, ideally removing them and re-importing them. Renaming, keywords, etc might be a bit ambitious.

As an example, I've recently been finding a bunch of images at random that have been corrupted somehow upon import/convert to DNG (or perhaps at a later date. Hard to say). The easiest solution is to delete them via LR, then import images from the appropriate backup folder (LR's import backup - RAW format, no renaming, etc). I then rename and re-keyword (etc) manually.
The most useful feature would be hunting out and grouping corrupt images.