Questions from "The DAM Book" 2nd Edition

[danaltick]

Peter, (responded to your post on previous page)

Here's a question about Data Validaiton: My hope is once I go with idImager and the new workflow outlined in the book with my buckets filling in the working files folders before transferring to the archive, I won't ever really have to touch my archive; except to add new buckets.  As you pointed out in the book, there are 3 components to checking the integrity of the archive: 1) directory structure, 2) file structure and 3) the image data itself.  Assuming I never had to touch the archive, would it be sufficient to just run something like Image Verifier to validate all 3 components?  I would assume that IV would not pass if any of those 3 components were corrupt; correct?  Might be dreaming to think I can do this though ;-).

Dan

[danaltick]

Nevermind, I am dreaming.  I completely forgot about the rebuilding of the DNG's after making PIE adjustments.  I guess the question still remains though: if you verify the file structure of all files on the disk does that imply that the directory structure is intact?

Dan

[peterkrogh]

Dan,
For a DNG archive, the best and simplest validation will be to check the embedded hashes.
Peter

[danaltick]

Peter,

Even with the embedded hash, do you feel it's still important to verify the file structure.  Afterall the file structure would contain everything else in the job jacket including the adjustments.

Also, I think I may know the answer to my question above: If you don't verify the directory structure, files may go missing and never get verified even though all the other files look good.

Dan

[peterkrogh]

Dan,
Validate completeness of the archive (must be done with a comprehensive catalog).
Validate the integrity of the files. (File structure and image data integrity) Done with DNG Converter (at the moment)

There's no real way to validate the currency or integrity of the PIE settings (or other metadata).  The biggest problems will likely be caused by human error (adjust in this catalog, then in that catalog, now have version drift and mismatch).  So you'll have to wrangle that stuff separately.

Peter

[danaltick]

Peter,

Makes sense.  Just wanted to make sure that you do recommend validating more than just the image data of the DNG's, which is all the hash will do.  It's also important to test for completeness as you stated as well as validate the file structure periodically.

Dan

[peterkrogh]

Dan,
Completeness of the archive is a second process that is managed by the catalog.
Since the only way to validate in bulk is with the DNG Converter, you'll be doing both hash-validation, as well as structural validation in the same process.
Peter

[danaltick]

Peter,

Now that I'm leaning towards IDimager, I should be able to keep my backup archive pristine (i.e. untouched).  If you recall, with EM2 I was going to periodically do a validated sync of the adjusted DNG's to the backup archive (instead of using LR the way you do).  No need to do that now; therefore, given that my buckets contain both Raws and Derivatives discussed previously, I should be able to just run Image Verifier on the backup archive to verify both file structure and image data of all files.  For the primary, I could still run IV on the derivatives.  For the DNG's, due to the adjustments, I would be better served probably just running a hash validation on the image data once either IDimager supports it, the DNG converter supports it, or some third party app supports it.  Sound like plan?

Of course I will use IDimager for completeness and System Mechanic for the directory structure.

Dan

[peterkrogh]

Dan,
Sounds very thorough.
Peter

[danaltick]

Peter,

Given your new workflow, how often would you say you do your output proofing before the DNG conversion versus afterwards?

Ahh, and here's another one, how often do you find yourself creating derivatives while your originals are still in the working files folder (e.g. a merged pano proof)?

Dan

[peterkrogh]

Dan,
I almost never create proofs from the proprietary raw - I pretty much always convert to DNG first. (Most of my output proofing is done with Expression, since the web galleries look like  the rest of my website.)

As to the pano-proofs, I try to make them before archiving the DNGs, and archive the pano proofs with the DNGs.  The one exception is at the end of the year when I am just putting everything away that is still in the working folder before archiving files from the next year.  In that case, I have just dumped everything to DNG and archived, and then hired Josie (nice t have a 15 year-old who knows Photoshop around the house) to make the pano proofs from archived DNGs.

Peter

[danaltick]

Peter,

Makes sense.  I would assume that it's common practice for you to make proofs and master files from your working file DNG's since they're cataloged along with the rest of the archive and may hang around in the Transfer_Me folder for quite some time waiting for the buckets to fill.  I would also assume that you do not wait to fill working file derivative buckets since that may take a very long time.

Dan

[peterkrogh]

>I would also assume that you do not wait to fill working file derivative buckets since that may take a very long time.

Not sure what you mean by that.  Derivatives hang around the working folder until the bucket is full (which, yes, can take a long time.)
Peter

[danaltick]

Ok, wrong assumption there.  I assumed you might want to get them to write-once media a little sooner; however, now that I think about it, you use separate drives for your derivatives and wouldn't burn them to Blu-ray until they filled anyway.  Since I combine mine, they would fill more quickly..

Dan

[danaltick]

Peter,

At what point do you typically remove your number-range folders: in the Landing_zone or the Transfer_Me?

Also, what about when you have multiple shoots on a single card.  I assume you either ingest those to different named job folders or you group them into separate job folders in either the Landing_Zone or Transfer_Me as well; correct?

Dan