Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
Print Page - With apologies - buckets AGAIN!

The DAM Forum

General => General Discussion => Topic started by: dkperez on January 14, 2008, 02:12:11 PM

Title: With apologies - buckets AGAIN!
Post by: dkperez on January 14, 2008, 02:12:11 PM
I may have misread in The DAM Book (although I just checked again and I'm not finding the specific section from which I thought I got my methodology),
but I'm still having some confusion about buckets and media......

Lets say I go out and shoot 10,000 images over a couple days.  At the end of each day, I take the images I've shot and put them into buckets -
WF_061, WF_062, etc, on day 1.  In these I have the original raw images captured in the camera.  As each bucket gets to the point where it fills up a DVD,
I start a new bucket.  So at the end of day 1, I may have WF_061_080110, WF_062_080110, and so on.  Concurrently, I'm taking some of these images
and running them through ACR, dumping out "corrected" jpegs, possibly editing in CS3, and putting them on my web site.  The jpegs get put into a
different bucket called DRV_061, DRV_062, etc...  The 061, 062, and so on matches the WF_061 so a jpeg image from WF_061 is in DRV_061. 

Once a WF bucket is full, I generate .dng files from the raw images in the bucket and put them in a RAW_nnn bucket where nnn matches the ID from the WF bucket...

On day two I continue - so if I ended day 1 with bucket 063 partly full I'd continue day 2 filling that bucket then moving on to WF_064.....  When I'm
done I have 10,000 images in 28 full buckets, numbered WF_061 through WF_088 and one partially filled bucket, 10,000 .dng files in 28 full buckets
number RAW_061 through RAW_088, also with a partially filled bucket, AND some number of jpegs and/or .psd files I created that are in buckets from
DRV_061 through DRV_088, but they may be almost empty.

SO, in my example, I have a bunch of WF buckets that fit neatly onto DVDs. Cool.......  BUT, I also have a bunch of RAW buckets that do NOT fit
neatly on DVDs - the .dng files are smaller than the original raw files, so maybe a 4.3 GB WF bucket only fills 3.6GB of RAW bucket.  AND, I have a
bunch of derivative buckets with jpegs and .psd files that ALSO don't fit on a DVD.  They may contain 800 MB of files (for example)... 

As they're current organized, I'd have to burn 84 DVDs - 28 WF, 28 RAW, and 28 DRV - to hold what I've shot.  What are the problems and
disadvantages of decoupling the final date of the RAW and DRV buckets from the WF bucket?  That way I could put as many .dng files in a
RAW bucket as would fit on a DVD but the WF bucket that contains image DP_080111_4567.NEF, WF_067_080111, may NOT be the same
bucket that contains DP_080111_4567.DNG since that image may fit in bucket RAW_066_080110.  I can see the most obvious possible problem -
there's no longer a 1:1 relationship between WF buckets and RAW buckets, but I'm only going to go back to a WF bucket if there's a disaster -
disk crash, loss of images in the RAW bucket, or equivalent.

Further (and this one I'm not as sanguine about), is there any reason I can't do the same thing between DRV buckets and RAW buckets?  I may only
use 10% of my raw images - often its more like 1% - so I'm going to have VERY empty derivative buckets...  Can I just fill those buckets up too?  In
my example above, I may only have 10 DRV buckets instead of 28.

From what I've seen so far as I'm cataloguing images, I could reduce the number of DVDs I'd have to burn significantly.  OR, am I supposed to have
been organizing this way all along, and I'm just a big donkey?

Title: Re: With apologies - buckets AGAIN!
Post by: BobSmith on January 14, 2008, 04:15:52 PM
The problem you're making for yourself is that you're trying to make derivative buckets names match the raw bucket names.  Don't.  The catalog software will keep track of which buckets contain which images.  Just fill up a derivative bucket until it hits 4.3GB and then move onto another.  What bucket number the files are going into means nothing.

I tend to keep files within a bucket inside of another folder than has something like date and place or client as the folder title.  If my catalog system ever completely crashed and burned along with all backups, I could manually (and painfully) sort things out using those folder names.

Bob Smith

Title: Re: With apologies - buckets AGAIN!
Post by: markpirozzi on January 14, 2008, 04:36:10 PM
In addition to what Bob said, another question has come up about what do you do with a derivative file from a previous bucket if you revise it.  The answer was the revised version of the file goes into the current derivative bucket you are working on.  This will further upset the 1 to 1 relationship of bucket names.

One of the major points of Peter's system is to make it as easy as possible to restore your files if something should happen to some of the originals or copies.  Consecutive numbering of buckets makes it easy to spot missing buckets - that's the main idea, just fill to ~4GBs and start a new bucket whether they are originals or derivatives.  If the raw file has a unique name and the derivatives files' name contain that unique name, you will be able to find the files relatively easily.


Title: Re: With apologies - buckets AGAIN!
Post by: dkperez on January 14, 2008, 06:20:49 PM
Thanks guys, you're telling me what I was hoping to hear.......  BUT, this leads to anothe question.....

I could swear that somewhere in The DAM Book I saw a figure or explanation that showed a 1:1
relationship between the working file buckets, raw buckets and derivative buckets...  Which is why
I set up that way...  Am I remembering incorrectly, or is that methodology useful for client
shoots but not so much for nature where you're using such a small percentage of shots?

The good news is that I can consolidate buckets very easily.

The OTHER good news is this whole process has given me a chance to consolidate some of the
redundant images AND get another look at some images I'd fogtotten about...

Title: Re: With apologies - buckets AGAIN!
Post by: markpirozzi on January 14, 2008, 07:02:15 PM
In the last couple of months in this forum Peter responded to the one to one question, so if a figure gives the impression that his system uses a one to one relationship, then the figure is misleading or interpreted wrong.  I believe there is at least one figure that shows folders inside of a "bucket folder" that have names corresponding to clients, subjects, etc.   Maybe you are recalling that figure(s).  This could be done within both the RAW and Derivative enclosing bucket folders, it is just that it would be very unlikely that numbers on the enclosing bucket folders would relate (ex. RAW_023 and DRV_023) for all versions of a photo if you fill buckets to a set size on the disc.


Title: Re: With apologies - buckets AGAIN!
Post by: dkperez on January 14, 2008, 08:51:06 PM
Well, I've now looked through the WHOLE book for the figure I thought showed the relationship and I'm not finding it.
So, its gotta be something I imagined.......

In any case, I'll just reorganize the RAW and derivative files and things'll be fine.

Title: Re: With apologies - buckets AGAIN!
Post by: peterkrogh on January 14, 2008, 09:32:49 PM
Thanks Mark and Bob for setting David straight..