Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
Print Page - Why make folders at all?

The DAM Forum

Software Discussions => Media Pro & Expression Media => Topic started by: jljonathan on December 12, 2008, 09:10:28 AM



Title: Why make folders at all?
Post by: jljonathan on December 12, 2008, 09:10:28 AM
I am setting up a Dam system using EM2 and was wondering why I should even use folders. If I build buckets for Raw and Dev, and keep keywords etc on all images synced, why use folders in the buckets? Just fill them until I make a DVD with images cataloged into EM2. Any thoughts on this?
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 12, 2008, 12:32:38 PM
Jonathan,
I was doing the same thing, when one bucket could containg images from just one shoot (I still found it marginally useful to keep separate shoots in their own folders.)

Now that I've moved to Blu-ray, it's unlikely I'll have a no subfolders in one bucket.
Peter


Title: Re: Why make folders at all?
Post by: jljonathan on December 12, 2008, 03:21:49 PM
Peter
The only reason I ask is that I often come back with only a few images on a card from shooting some occasional event or person, and the folder that gets made only contains a few shots. I have a number of these on the system and in EM2, and thought that maybe having all these folders wasn't really necessary. I do understand your point of keeping things in folders though. Maybe I should rearrange some of these items that are in in separate folders,  consolidate based on subject or topic etc.  and forget about dating folders. It's the dating that means that I create a new folder for what might be only a few files. What do you think?
Also, what is the capacity of the bluray and the cost?
Thanks again
Jonathan


Title: Re: Why make folders at all?
Post by: billseymour on December 12, 2008, 04:57:16 PM
Peter-
Isn't the folder concept also based on DAM book idea of having 'chunks' of images in folders the size of the physical media you are backing up to. (Hmm, ending a sentence with a preposition...)

So folders of 4gb for saves to single sided DVD, etc. Then as you suggested, when one 'scales up' to larger capacity (like Bluray), one might keep folders of Blu-Ray size (is that 28gb? I haven't kept up), and the older folders just get tossed in intact (so BluRay001 will contain DVD001, DVD002, etc).

Personally, I find it useful to have the 'simple chunk' of a folder when doing saves. We've discussed elsewhere the question of 'updating' the physical media saved periodically, for instance, if one has totally revised one's keywording and wants to physically save the synched media. In this instance, I find a few folders more useful than a no-folder setup.

(I understand that if you are talking about 'a folder for each job', that this seems unnecessary and too cluttered for a DAM system).
--Bill


Title: Re: Why make folders at all?
Post by: jljonathan on December 12, 2008, 07:30:14 PM
Bill
Yes I am referring to many folders within the RAW_001 folder that gets backed up to DVD.
Folders like:
Car Show_2007 , with maybe 15 images
Mike_2006 , with maybe 3 images
and so forth, all adding up to many folders, some with very few images in them. That's whyt I'm trying to decide to either continue with it this way, if it seems that this is the recommended way to go, or, rebuild the folder system so that there are fewer, (I mentioned maybe consolidating), or no folders (sub-folders) at all. I'm looking for suggestions and recommendations.
Thanks again
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 12, 2008, 11:20:50 PM
Jonathan,
I tend to lump lots of small shoots together in a folder "misc" instead of having 20 different ones in a bucket.
Peter


Title: Re: Why make folders at all?
Post by: BobSmith on December 13, 2008, 05:21:39 AM
in my view, part of the reason for a few simple meaningful folders is as a fail-safe last resort to find something if the catalog structure and backups all go south.  My folders are all named with the date (YYMMDD) and a key descriptive word or two... most of my archive is commercial work so its the name of the client.  If I had to, I could do system level searches for certain dates or client names to locate groups of relevant images. Most of my DVD size buckets contain no more than maybe four or five folders... many contain only one or two.

Bob Smith


Title: Re: Why make folders at all?
Post by: jljonathan on December 13, 2008, 07:05:13 PM
Peter and Bob
Your responses give me some guidance as to how to eliminate the excess folder situation here. The misc folder is good one. Maybe I will create several of these based on date ranges.  One additional aspect is the DRV folder situation. When a file goes into PS and then is saved in a DRV bucket, do you do the same thing? Otherwise, I will end up with many folders in there. How do you handle the DRV folders?
Jonathan


Title: Re: Why make folders at all?
Post by: johnbeardy on December 14, 2008, 02:57:08 AM
I feel the answer would be the same if you were to ask whether it's worth bothering with any text in filenames. A hard core meta-pedantic view would say filenames only need the date (if that) and some numeric code, and would dump those files into big buckets. In practice, you're not always going to be examining the archive through your cataloguing application - you'll be using Finder/Explorer. And just as text in filenames helps us poor humans, so does subdividing into folders.

FWIW I have one or more folders per day. If a shoot's keepers exceed a single DVD bucket, I'd add a letter - eg 081214A Jones Wedding. It's not much work to make them, and if you find it convenient, why not? if you end up wasting a lot of time, go hard core.

John


Title: Re: Why make folders at all?
Post by: peterkrogh on December 15, 2008, 08:26:57 AM
Jonathan,
I generally make folders for each derivative project.  Since the files are much larger per image (Master files routinely get to be several hundred MBs each for me) even a bucket full of misc generally does not have too many folders. Most projects include a number of Master Files, so average size is probably several GB.
Peter


Title: Re: Why make folders at all?
Post by: Dierk on December 15, 2008, 09:35:02 AM
Simple answer from me [in two varieties]:

a) easier to find photos
b) always good to have a back-up system or 'Plan B'


Personally I go for a semantic system with folders, that is, I am more interested in an intuitive, content-based model. I rarely have the need to search for dates; I am much more inclined to look for, say, a 'fallow deer' photo or something from 'Upper Lusatia'. Expereience shows that people in general function that way, abstracts [like dates] aren't in the human nature.

I also find, over and over again, that it is often quicker to find a photo in a system using descriptive folder/file names [even on the lowest level] using ones brains than by abstract names, metadata and a DAM. This is clearly needed when the DAM, for whatever reason, fails.


Title: Re: Why make folders at all?
Post by: peterkrogh on December 15, 2008, 10:19:32 AM
Keep in mind that embedded metadata is at least as durable as folder names, even if the particular catalog you are looking at were to totally fail.  If you depend on the folder name to lead you to "Fallow Deer", the system will break if the image is moved outside the folder.

While you can use a folder structure in a way that gathers related content, it plays real havoc with the backup strategy, and is going to be really limited.  Are you organizing images by subject, client, or how god you think they are?

Peter



Title: Re: Why make folders at all?
Post by: jljonathan on December 15, 2008, 01:07:00 PM
Well, it seems that there is lots of room for various approaches to the use of folders in organization. Dierk takes the content approach to finding his 'Upper Lusatia' photos (thanks for the reference, interesting area settled by the Celts). I certainly understand the desire to have content as a reference and was using a lot of content as my initial method. Peter takes the approach that metadata can reference any organization you choose, and I think that this is a very powerful method that is now what I am trying to transition into. Any of the information from dates, content, keywords etc etc can be embedded and searched as a locator, both in EM2 or Bridge. What I was trying to get at was the problem for someone who does not do commercial jobs with hundreds of images for a single shoot, but someone who, as a hobby, takes shots and then ends up with too many folders holding few images that identify that specific image by content in the folder name. Additionally, adding to the abundance of folders in the DRV buckets also. I was looking for some guidance for an effective way to utilize the metadata as a primary organizational/search tool, but also have a simplified and more limited folder structure within the RAW ans DRV buckets. The Misc folder is one option for dealing with situations when there is only one or two images within a folder named for the content. I would still be interested in any others, short of doing away with folder altogther.
Thanks again to all
Jonathan


Title: Re: Why make folders at all?
Post by: Dierk on December 16, 2008, 02:30:02 AM
If you depend on the folder name to lead you to "Fallow Deer", the system will break if the image is moved outside the folder.

True. And often times, as I am on record, it isn't really easy to define the subject of a photo as clear cut. A shoot of, to stay in my vicinity, Hamburg's harbour birthday might best be put into Hamburg->Harbour->Birthday[->YYY]. But what with those photos depicting old schooners, u-boats and submarines, the newest ocean-crossing racers or the largest cruisers? Wouldn't I look for them under Movement->Water->Vessels[->sailing|cruise|sports|military]?

It gets worse when you factor in probable uses; a photo used journalistic today [Hamburg's harbour birthday] may be advertising for sweets tomorrow, a postcard next year etc. That is the subject changes by use.

That is why DAM is so important, to create several layers of meaning for a photo. You can put it the other way round if you like: to manage the possible layers of meaning of a photo. Do not use one system only, folders or DAM, use both to get the best of both worlds.


Peter, I actually use my folder structure to organise the hotos by content. I try to make this as concrete as possible; abstractions like client or perceived value [aesthetic, economic, morally] do not play a role unless client and content coincide [portrait, commercial still life]. Just as with the subject mentioned above, aesthetic, moral and economic value change with time and target/usage.


Title: Re: Why make folders at all?
Post by: peterkrogh on December 16, 2008, 07:23:24 AM
Dierk,
Your post remonded me of one of my favorite imaginary visuals - when I was touring Hamburg, the Harbor birthday was described - "imagine thousands of hamburgers out in the harbor in their boats". To this American, it conjured an image of a harbor full of Mayor McCheese characters in their boats bobbing around n the twilight. (Which probably has *you* scratching your head...)

But on to the point.  When you group by subject, how do you manage to keep your backups in sync?  Are you using a sync program that adds all the new  harbor birthday photos to the harbor birthday parent folder? I assume that this means no write-once media in the backup scheme, or at least no direct correlation of optical media to archive structure.
Peter


Title: Re: Why make folders at all?
Post by: Dierk on December 16, 2008, 07:45:29 AM
MirrorFolder

With that program I sync on the fly for all of my working HDs and partitions. My second back-up is to DVDs via ArchiveCreator, essentially replicating the organisation with the caveat that it's not 1:1. If I need to find anything on my DVDs I have to use my xMedia DVD catalogue or go disc by disc. I am constantly thinking about doing a full back-up once a year from HD to DVD but then leave it out of laziness. When Blu-ray finally gets to computers at reasonable prices it will be done.



Tangent starting.

When I was younger I was much more afraid of losing data. I got over it, not least because it happened to me several times. there's nothing like a completely save system. Full stop. Not even with film.

While I do have photos I'd rather not lose - almost all of them family pics - I reckon I can easily do almost anything again. Or it doesn't matter really [for journalistic pics]. I've never been a collector type, don't even understand them, I am more of a hunter.


Title: Re: Why make folders at all?
Post by: peterkrogh on December 16, 2008, 11:54:18 AM
Dierk,
The differences between our approaches are certainly informed by what we are looking to get out of the storage system. 

Folders can only do one thing at a time very well. For you, the most important thing is to group by subject.  For me, it is to protect the images from loss. It's important to know yourself and your goals.

Peter


Title: Re: Why make folders at all?
Post by: jljonathan on December 16, 2008, 12:06:43 PM
Interesting discussion of organizational approaches. Thank you.
But now, I would again ask for some input:
What I was trying to get at was the problem for someone who does not do commercial jobs with hundreds of images for a single shoot, but someone who, as a hobby, takes shots and then ends up with too many folders holding too few images. And then, identifying that specific image by it's content in the folder name. And in addition, adding to the abundance of folders in the DRV buckets later, when masters etc are made of these images. I was looking for some guidance for an effective way to utilize the metadata as a primary organizational/search tool, but also have a simplified and more limited folder structure within the RAW ans DRV buckets. The Misc folder is one option for dealing with situations when there is only one or two images within a folder named for the content. But after a while, that Misc folder could end up loaded with strays. I would still be interested in any others suggestions, short of doing away with folders altogether.
Thanks again to all
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 16, 2008, 03:08:29 PM
Jonathan,
If you have lots of little groups of files, then metadata starts to become even more important.  The folder structure becomes unwieldy even faster.

Try using keywords or catalog sets to do the organizing.  See how that feels. 
Peter


Title: Re: Why make folders at all?
Post by: jljonathan on December 16, 2008, 05:20:12 PM
Peter
That is what I am leaning toward, and I do appreciate your assistance. The keywords and catalog sets would of course work correctly in EM2. How do you get the catalog sets show up correctly in Bridge, etc? I just tried making  a catalog set of a few images, used the sync annotations in EM2 but I don't see any indication of it in Bridge. When I make keyword changes in EM2 and sync, they do show in Bridge. Without catalog sets showing up somewhere else, I would be a little hesitant to only work with the keywords.
And, if I take your advice and use keywords/catalog sets for my organizing, I still would like to know what you suggest for how I handle folders. I do have lots of little groups of files in lots of small folders. Do you suggest abandoning folders altogether, or some other option in between, like sticking with misc or dates, even though this would end up with folders filed with many disparate items. Almost like sub-buckets.
Thanks again
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 16, 2008, 09:27:11 PM
Jonathan,
Not sure why you need to see the information in Bridge - that would help to understand what is the right tool.

As to the organizing, I'd say to use sets or keywords to bring images together.  I'm guessing that many of the small groups are recurring subjects (maybe easy to group, like a one-off picture of my kids - or more esoteric, like photos I make of my hotel rooms).  In any case, metadata can bring images together that would never be grouped naturally in a folder, because they were shot so far apart in time.
Peter




Title: Re: Why make folders at all?
Post by: jljonathan on December 17, 2008, 12:37:51 AM
Peter,
First off.  I guess that doing away with the folders based on content etc., means that I would be organizing completely by keywords and catalog sets in EM2. Fine with me. Except the slight anxiety of something going wrong in EM2, or future versions of EM etc. Bridge would be my only option, and there I would only have keywords because the sets would not show up to take the place of folders with content names, which I would no longer have there to fall back on. If the catalog sets were available in Bridge, that would at least ease the situation.
Second. Yes, I do have many small groups of somewhat recurring subjects, there are also some large folders holding larger shoots. Metadata would be perfect as an organization tool. Both keywords and sets combined. Are you then suggesting that I do away with folders altogether or a middle option, such as the misc folder idea for folders with few files (but as I mentioned, this misc folder could end up  a hodge podge after a while. What would you suggest.
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 17, 2008, 07:44:52 AM
I am not suggesting doing away with folders altogether.  I'm just saying that you don't need to depend on them for content-based organization.

Expression Media is not going to magically disappear one night, never to be seen again.  You may, however, decide you don't want to use it any more, and need to get your information out of it.  Transferring the organizational structure from catalog sets to , say hierarchical keywords is not something that takes all that long.  As a demonstration, I recently looked at what moving my 40,000 image 2006 catalog from EM to Lightroom would look like - it took less than a day to do the full migration. (Select all images in a set, and assign a keyword for that set).

I'd say that your folder structure is *much* more fragile than embedded metadata as a carrier of content information, even proprietary XMP fields like Catalog Sets (if it's written to XMP, it can be read by Bridge.)

I think that you are way over-thinking this one.
Peter






Title: Re: Why make folders at all?
Post by: jljonathan on December 17, 2008, 12:23:29 PM
Peter
You are right about the over-thinking. I guess since I'm using EM2, and by using keywords, I'm already not completely dependent on content based folders. I have not yet tried using catalog sets in EM, so I think I will give that a go and see how that works out. Is there a way to write the sets to XMP that can be read?

I still would like to reduce the amount of folders holding only a few images. Could you offer some suggestions.

Are you planning to make a switch from EM to Lightroom?

Thank you again
Jonathan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 17, 2008, 01:35:29 PM
When yu sync annotation in EM, you get the option of writing Catalog Sets to XMP. THey're not easy to see (File Info>Advanced, unless you have Beadsworth's panel).  But this does, in fact, write the data to the file.

As far as the folders thing, I'd just move the images into larger misc folders.

And as far as moving to LR, it will be interesting to see who gets to a better alternative than the current EM first - Adobe, MS, someone else?  Right now, the catalog part of LR is unsuited to my on collection (catalogs too large, other stuff outlined in the LR part of the forum), but they are making progress fast.

Peter


Title: Re: Why make folders at all?
Post by: jljonathan on December 17, 2008, 11:24:05 PM
Peter
Thanks again for all you suggestions and help. I tried the xmp panel and it does work in Bridge.
Jonathan


Title: Re: Why make folders at all?
Post by: Doug on December 19, 2008, 01:37:52 AM
I'm probably in between Jonathan and Peter in volume. I can shoot a few thousand images in a month, but not every month. I find folders by either subject or location useful for my lowest level. I have Year then Year_Month (2008_12) then any name that'll help me find stuff later. Its really irrelevant to the DAM, but it helps me when I'm looking without a DAM app. If my shoot is a certain trip stretched over a couple weeks, it'll all go in a folder named for the place, for example. Just jogs the memory.

With Toast (and probably others) able to span many DVDs, automatically creating 4.3GB chunks from an arbitrary amount of initial data and then burning them in succession, I don't personally find any use for breaking things into buckets.

-doug


Title: Re: Why make folders at all?
Post by: peterkrogh on December 20, 2008, 01:26:50 PM
Doug,
Span-burning does help, but there's one thing you need to be careful about.  You can't burn the month until you've processed everything you shot in that month.  I find that oftentimes I don't get around to clearing my backlog until months after some of the images have been shot.

Of course, if you ingest directly into a month folder (as opposed to using a Working folder to hold images until ready to archive), then you should be able to span-burn at the end of the month, and be sure you have everything burned.

Are the disks that are produced by Toast in such a form that it would be easy to reconstruct the archive, and know that everything has been restored?
Peter


Title: Re: Why make folders at all?
Post by: danaltick on December 20, 2008, 10:03:40 PM
I remember talking about buckets with people a few years back when Peter first wrote the book.  It just seemed overly complex to people; but it's really very simple if you think about it.  As long as we use write once media for permanent storage, I just don't see a better way to handle it.  Of course DVD's "are" beginning to run their lifespan now with larger megapixel cameras, but Peter has already adopted Blu-ray; and I'm not far behind.  Buckets are really a very simple step in the workflow with minimal overhead.  I actually have a simple bucket template on my archive drive that I just copy, paste, and increment every time I need a new bucket.  I've even gone back to a flat folder scheme inside each bucket.  Given that I usually have no more than a half dozen to a dozen folders in any given bucket, it just makes things simpler.

Dan


Title: Re: Why make folders at all?
Post by: peterkrogh on December 21, 2008, 10:18:52 AM
Dan,
As I've been writing the new book, I really wish that I could do away with buckets, but we're just not there yet.  It makes things more complex, particularly for the Lightroom user where folder structure is one of the tools that the programmers expect you to use for informational organization. 

The biggest problem with non-bucketed approach, is that it either means write-once backups are not used at all, are not complete, or will not be usable to restore the archive with confidence in the event of drive failure.

I know I'm a magnet for data-loss stories, but I've seen (on my own drives) and heard about way too many stories of data loss to rely on hard drives alone, at least until DAM software is able to validate completeness and integrity of the archive. With hard drive-only storage, it's way too easy to have the corruption on a primary drive be propagated to the backup drives without knowing it. There's the very real possibility of loss of *everything*. (LOT4 tape backup can also protect without buckets, if you have a spare $10,000 lying around).

Anyway, one day it won't be necessary (or if you re like Dierk and can make peace with the possibility of data loss - that day is today.)
Peter





Title: Re: Why make folders at all?
Post by: danaltick on December 21, 2008, 11:05:55 AM
Peter,

I share the same sentiment.  I'm not even a full time professional (working on it), but given my years of experience with computers, backups, and media in general, two hard drive copies would just not let me sleep at night; especially if my digital assets were extremely valuable.  And even more so if they were my sole source of income.  You just cited another complication with Lightroom.  I do like being able to just drag my new buckets to iView/EM and then using catalog sets for any further organizaition; not to mention multiple catalog capability.  With Bridge and ACR continuing to improve, I just can't justify switching over just yet.  I am interested to read your perspective on this in your new book.  Looking forward to it.

Dan