Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
The blurring line between originals and derivatives: workflow implications
The DAM Forum
Welcome, Guest. Please login or register.
August 11, 2020, 06:21:52 PM

Login with username, password and session length
Search:     Advanced search
28033 Posts in 5147 Topics by 2904 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  General
| |-+  General Discussion
| | |-+  The blurring line between originals and derivatives: workflow implications
« previous next »
Pages: [1] Print
Author Topic: The blurring line between originals and derivatives: workflow implications  (Read 6939 times)
Arlen
Newbie
*
Posts: 31


View Profile
« on: February 13, 2009, 12:21:03 PM »

Since reading Peter's book a couple of years ago, I've been employing a workflow similar to the one he recommended. It seems to me that a central concept was to differentiate between original and derivative image files. Originals are processed, archived and backed up, and then left essentially unaltered. Avoiding any "resave" steps minimizes the chances of corruption of the originals, and assures that you will always be able to go back to a good file if need be. Indeed, if you back up to DVDs, the originals can't be changed, which is one of the safety features of that strategy. So all subsequent modifications of such a file become derivatives, and each one is saved as a new version in the current working "bucket". It all seems very logical and practical.

But now I find myself increasingly perplexed about how to best handle the ever-increasing tendency to make image modifications in metadata rather than in pixels. The advent of DNG file wrappers, as well as ACR and Lightroom methodology means that the majority of my changes occur in metadata. Do I save the modifications within the original DNG (or TIFF or JPEG, for that matter) file? If I do, I violate the rule of leaving original files unchanged, and archiving becomes very problematic. On the other hand, if I make a copy which becomes a derivative (e.g., filename.dng ---> filename-b.dng), the size and speed economy of metadata edits is completely lost.

So, how do some of you handle this situation where the difference between an original and a derivative image file becomes ever more blurred?

Arlen
« Last Edit: February 13, 2009, 12:26:40 PM by Arlen » Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #1 on: February 13, 2009, 01:01:38 PM »

Arlen,
Great question, and one I address in the new book.

I see a couple options.  What I do is to send the reworked images through a Lightroom catalog for just that purpose, and make sure to keep it well backed up.  In the event of loss of the primary archive, you could use that catalog to update all reworked images with the new settings. 

You could also decide to push the new settings into the backup files (perhaps easiest by syncing the primary to the backup, with your backup software set to update any changed files.)  This would offer more protection for the new edit settings, but at the expense of security of the backup to some degree, since it would also be possible to overwrite valid files on the backup with corrupted ones.  If you had virus or disk errors or transfer errors, the very files you want backups of would likley also be corrupted on the hard drive backup.  Of course, if you have DVD or Blu-ray backups, you'd still be protected, as long as those were readable.

Ideally, the catalog would wrangle the changes to image settings, as part of your other metadata preservation.  idImager does this, although I have not tested it to make sure it works.

Does that help you make a decision on how to move forward?

Peter

Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #2 on: February 15, 2009, 06:49:13 PM »

Thanks Peter, those approaches do make sense; although neither one seems ideal. Currently I'm using your second option--pushing modified files into the backups. But I worry about the potential problem that you mentioned: corrupted files migrating onto the backup drive. If you make frequent changes to the "originals" on the primary drive, then you must also frequently migrate those changes to the backup; which of course increases the chances of the backup becoming corrupted somewhere along the line. If the second backup set is on DVD/Blu-ray, the originals should be protected from overwriting by corrupted modified files. But then there is only one backup of the modifications. (In my case, I decided not to use DVDs, but rather an additional hard drive backup. I understand the risk there, but decided that for my purposes the additional safety of the DVDs was not worth the hassle of burning and storing them. I do try to avoid overwriting any files on the secondary backup drive, but just add new files to it periodically, for what that's worth.)

I hadn't thought of using Lightroom as a strategy for saving all the develop modifications. I assume then that you uncheck "Automatically write changes into XMP" in the LR Catalog Settings? I'll have to mull over that approach. Right now, I do some of my modifications in Bridge/ACR, and some in LR. It seems that if I were going to rely completely on the LR catalog for saving develop edits, I would need to avoid using ACR on a file after the original tweaks and archiving, and I would need to bring every file into LR for subsequent work. Is that what you are doing? [Edit: Upon reflection, I guess there's no compelling reason to prevent LR from writing changes to the original files, as long as those changes are not pushed into the original file backups.]

I think you are right that the ideal situation would be for Expression Media to save the develop metadata in its catalog; then you could just do all subsequent work from there, rather than having to go to another program. Maybe in a future version, EM could incorporate the ability to open a file in LR or ACR, and save the develop metadata in its catalog rather than elsewhere? I don't know much about IDimager, but I took a quick look at its manual, and it appears to me that the edits would have to be made using that program's own image editor (not ACR or LR) in order to have it save the develop metadata in its catalog.

When is the new version of your book coming out?
« Last Edit: February 15, 2009, 07:30:56 PM by Arlen » Logged
Photo_op
Guest
« Reply #3 on: February 16, 2009, 04:36:54 AM »

Peter - probably could wait for the book update, but a couple of questions if appropriate? I use a DNG workflow. My collection of photos is currently @15,000, folder & bucket method. I currently manually save metadata and adjustments to "originals" (backup to disk).

1. I have lost the answer to this question in the fog of learning LR. I assume Print uses info from the catalogue in order render all adjustments made to the originals and then print. Yes/no?

2. If the answer to #1 is yes, are you suggesting, especially in my case, where there is potentially only one catalogue, that I could simply NOT update my originals (by keeping any changes in the Catalogue) and therefore-
     a. significantly reduce potential corruption issues to my originals, and
     b. make any photo movement from folder to folder within LR, so it "remembers" file location. I very seldom move photos, since I use the bucket method.

thanks in advance-

Dave
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #4 on: February 16, 2009, 08:53:20 AM »

Arlen,
You outline a practice that starts to edge to uncomfortable risk for me - using hard drive only, and updating the backup.

You open yourself up here to a higher level of risk because of the combination of these two factors. You greatly reduce your protection against virus, directory corruption, transfer error, or other program error that screws up files if you update the backup.  You also lose protection against those many of those same hazards by forgoing write-once backups. And the files that are most likely to be affected, should there be a problem, would be the valuable ones that you have decided to rework.

I explore this concept of risk multipliers to some degree in the new book (due out in April). It's something to watch out for.

If you want to update your backups with current versions of the files, I suggest that write-once should be part of your system. Alternately, you could run ImageVerifier ( or just run the DNG converter) on the primary before you update the backup - this would let you have a much higher level of certainty that you are not overwriting good files with corrupted ones.

Peter

Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #5 on: February 16, 2009, 03:21:27 PM »

Thanks for the feedback, Peter. I share your concern about the potential for corrupted files migrating to the backups, if the backups are continually updated; that's why I've been worried about the "originals as derivatives" issue pushing me in that direction. My situation may not be quite as risky as it first appears though, because I've been keeping two separate backup drives, and only updating one of them with modified originals. And I have been using ImageVerifier to check the files. Also, my really mission-critical files (in particular, several hundred images that went into a couple of books) were burned to DVD as insurance. But while I would hate to lose ANY files, I don't depend on most of them for a living. If I did, I would certainly reduce my risk to the minimum by adopting write-once media backups for all my files. Anyone who would be devastated by the loss of files should follow your example, not mine.

For most of my work, I think that your suggestion of using LR to save changes to originals, and then making a couple of backups of the LR catalog, will be a good way to go. Then I won't need to make changes to the original DNGs, nor push them into either of the backups. But I'll read the new version of your book when it comes out; maybe I will change my mind about how much risk is acceptable.

Arlen
Logged
David_E
Newbie
*
Posts: 20

Deep in the Enchanted C-Stand Forest


View Profile WWW Email
« Reply #6 on: February 17, 2009, 06:40:47 AM »

Under the guidance of the "keep it simple" principle, it seems to me that keeping a DAM catalog AND a Lightroom catalog (and the accompanying backups of each) is an unnecessary complication.  Don't we rely on our carefully established hard drive and DVD backup procedures specifically because it gives us the freedom and confidence to make amendments to original files?  When I adopted Peter's (terrific) techniques in the book I chose to balance the advantages of "live and local" with the hassles of building and managing my own DIY data center.

David_E
Logged

havezet
Full Member
***
Posts: 180



View Profile WWW Email
« Reply #7 on: February 18, 2009, 02:41:59 PM »

I don't know much about IDimager, but I took a quick look at its manual, and it appears to me that the edits would have to be made using that program's own image editor (not ACR or LR) in order to have it save the develop metadata in its catalog.

Hi Arlen,

That is incorrect, you can create version sets in IDimager that contain any image you like. If you create an edited version with IDimager's editor then the saved copy is automatically added to a version set. If you create a version with a 3rd party tool then you can either add the file to a version set by hand, or use the Version Detection Wizard to find derivatives based on file name conventions or metadata matching.

Hert
Logged

Author of IDimager
http://www.idimager.com
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #8 on: February 18, 2009, 02:55:25 PM »

Thanks Hert, that is useful information.

Arlen
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!