Working Folders, Flags...Workflow

[Dale]

Has anyone here looked at the pros and cons of ingesting images into a permanent directory and then tracking the workflow using a keyword flag rather than working folder location? (This thought came from the discussion in the book (pp. 259, 287) about sending images to permanent homes right away if you use LR and are a single-computer user. I will need to use my laptop some, but still thought couldn’t flags substitute for folders and make the process work equally well on single or multiple computers?)

I’m just thinking through alternatives as I design my workflow. Thoughts?

Dale

[peterkrogh]

Dale,
Very good thinking - ultimately, moving images through folders is kind of a blunt instrument, compared to using metadata-based tools.  And if the tools were created well, you could make use of lots of intelligence that's already in the files.  Beardsworth, for instance, has some very clever smart collections that let you see what images have been edited, how fresh or old the edits are, and other groupings that can be based on develop metadata.

Unfortunately, for my own workflow, this is not really workable. One issue is that I must have drive transfer be a part of my workflow - I frequently download in the field, and my main workstations and not the same computer that serves up the archive. Since Lightroom does not have integrated tools for validated transfer, the easiest way to keep this straight (for me) is to use a folder-based pipeline.

If you can ingest directly into the permanent home, then using metadat to track editing would be preferable, even in the unfinished state that Lightroom offers.
Peter

[Dale]

Peter,

Thanks for those thoughts. When in the US I always download on the desktop computer that would hold the archive. So it sounds like a keyword flag process could work well if that's the only situation.

However, when I am overseas several months at a time, I need to do two things. The first is download to an external drive for a temporary backup. That shouldn't affect this process, since I'd still ingest the files on the desktop when I returned home. The second is more of a problem in that I usually need to use some of the images (generally less than a hundred of them, however) while I'm gone. I hate to design a workflow around a small number of images, especially since any editing I do on my laptop is less than optimal because of the screen. I've thought about a temporary "offline" workflow for use in those limited instances. That complexity, though, may reduce the benefits of the workflow in the first place. However I'm not sure how practical it is to make a workflow less efficient to accommodate a fairly limited number of situations.

Thanks for your help in thinking this through!

Dale

[johnbeardy]

I'm not a fan of using keywords to monitor workflow. They're meant to describe the image contents, and even if you're careful about clicking the "do not export" check box, they have a way of finding their way out of your catalogue - again, they're meant to be portable and universal.

Instead, LR offers collections. So you might have a set of collections such as Unedited, Partly Edited, Full Edited etc which function just like folders, and Peter is referring to my Workflow Smart Collection You might even use coloured labels as another indicator of progress.

John

[peterkrogh]

Good point John.
I was careful to say "metadata" to track workflow, rather than "keywords". And Collections /Smart Collections are a good tool for this.  It gets problematic if you need to move through multiple computers, however, sinc ethe Collections can't be written back to the files. Labels could work for this, depending on what other uses you want to put them to.

And the problem Dale outlines - multi-computer workflow - is just the kind of situation where I'm more comfortable relying on folders to track. Once you introduce the need to track file transfer into the workflow, you start to need to use tools that can be seen by the OS, organizational, and transfer programs. Metadata tools simply fall short for these tasks.

Peter

[johnbeardy]

"It gets problematic if you need to move through multiple computers"

Peter

That's where instead of moving files I take advantage of Export as Catalog and Import from Catalog, as they carry over collections too. Of course, it's another matter if you mean moving between applications.

John

[danaltick]

Interesting topic.  This never really occurred to me.  In theory, for a single computer I think you should be able to do this with iView/EM with your raw files (i.e. ingest to permanent home and catalog), then after the conversion delete the raws from both the catalog and archive then import the new DNG's?  Of course this would require two imports of the metadata, a possible time consuming task for large shoots.

Even though you might could do this, having a separate working files hard drive certainly has its advantages (e.g. offline swappers and separate bucket management). However, it might be sufficient in this case to just have a "Transfer_Me" and a "Transferred" folder.  The rest of the pipeline could be implemented with metadata.  I'm looking at possibly doing this with IDimager.  With its built-in Downloader, Raw to DNG converter script, and separate customizable labeling for different stages of the workflow, I think it lends itself well to this approach.  Unfortunately it doesn't yet have transfer validation, but I'm working on that ;-).

Peter, if you know of any drawbacks or gotchas I need to be aware of with this approach, please let me know.  I would think this could also be applied to the multi-computer just by adding the exporting/importing of the metadata along with the transfer validation.

Dan

[havezet]

I'm looking at possibly doing this with IDimager.  With its built-in Downloader, Raw to DNG converter script, and separate customizable labeling for different stages of the workflow, I think it lends itself well to this approach.  Unfortunately it doesn't yet have transfer validation, but I'm working on that ;-).

Dan, what do yu mean when you say that IDimager doesn't have transfer validation? The integrated Downloader will binary compare the copied files with their original.

Thanks

Hert

[danaltick]

Hert,

Thanks for chiming in.  Yes I know the Downloader has validation, which is definitely a plus at ingestion.  What I'm talking it about is the validation needed later when transferring the working job folders to their permanent home in the primary live and local archive.  In a (for lack of a better term) professional DAM workflow this would be a separate hard drive or set of hard drives from the working files hard drive.  What would be ideal would be the ability to do a validated transfer of the working files job folders from the working files hard drive to the primary archive hard drive using the Media Browser.  I believe IDI supports validation on a file basis, but not on a folder (or directory) basis.

Maybe a justification here is warranted.  After ingestion the images are typically visually inspected during the rating, editing/adjusting, and metalogging phases.  If the ingestion is corrupt this will typically be discovered during this visual inspection; however, when the time finally comes to transfer the final images to their permanent home in the archive, this transfer is usually not followed by another visual inspection, which would incur additional overhead in the workflow.  However if the transfer is a validated one, no inspection would be necessary, and the photographer can rest assured his/her images have made it safely to their permanent home.  Peter talks about this in his latest book.  Hope that helps.

Dan  

[peterkrogh]

Dan,
I don't see any particular gotchas with using metadata for status.

Hert/Dan,
In another post, Dan indicates that a registry edit can be used to create a data validation transfer with idI. oes this work for *any* transfer by idI, or just for initial download?
Peter

[danaltick]

Peter,

Any transfer.  Here's the discussion I had with Hert http://forum.idimager.com/viewtopic.php?p=67445#p67445.

Dan

[danaltick]

I don't see any particular gotchas with using metadata for status.

On second thought, I could see how a metadata pipeline might work for an application like LR, but once you throw in a file browser like Bridge, the Adobe DNG converter, and SyncBack which are all folder based, I think the metadata workflow really breaks down.  Even with Idimager it's not really feasible to implement a secure DNG workflow due to the need for folder based transfer validation, post DNG conversion after ingestion, and Bridge/ACR editing.  Also, even with LR I would think you would want to maintain a separate working files hard drive for backup purposes during the working phase.  Kind of lost touch with how LR does it's file management.  Been away from it too long.

Dan

[MikeBaynes]

Dale,

I'm coming late to this discussion.  I'll start by saying that I'm an amateur, with relatively low volume, so these comments may or may not apply in your situation.  I ingest (using II Pro) directly into my final file structure.  Then when I import into Lightroom, I have it set the purple color label on all files.  I don't use that color for anything else, and it gives me a visual indication of which images have not been worked on.  I'm a little bit casual about post processing, so if I go away and come back some time later, the color really helps.  As soon as I process an image I clear the color or set it to Red for rejects, etc.  My keepers end up with no color.

Where this scheme might break down would be if I were to go back to a "purple" image and make adjustments after I had already backed it up to an archive bucket.  However, I'm not doing that yet -   my database is still small enough that I just back up the entire thing periodically.

Hope this is helpful,
   - Mike Baynes

[danaltick]

Mike,

This is a perfect example of where it is advantageous to keep your working files in a separate location/hard drive from your primary archive and backup the working files hard drive daily and offine/offsite as needed.  Swappers is a great way to do this.  Once you have filled your DVD/Blu-ray sized buckets on your working files hard drive you can then perform a validated transfer of those filled buckets directly to your primary archive.  Peter, discusses this at length in his new book.  If you don't have a copy, I highly recommend it.

Dan

[MikeBaynes]

This is a perfect example of where it is advantageous to keep your working files in a separate location/hard drive from your primary archive and backup the working files hard drive daily and offine/offsite as needed.  Swappers is a great way to do this.  Once you have filled your DVD/Blu-ray sized buckets on your working files hard drive you can then perform a validated transfer of those filled buckets directly to your primary archive.  Peter, discusses this at length in his new book.  If you don't have a copy, I highly recommend it.

Dan

Dan,
   I have read the book - both versions - and I understand and completely agree with the point you made.  I don't thnik the use of a bucket approach should have anything to do with whether one uses color labels in liew of transfering working files among a series of working folders.  I was just mentioning that I personally haven't found an urgent need to take that next step into archive buckets.  My database of a few thousand images is still too fluid to consider any of my images to be "archivable".

Thanks,
   -Mike