Questions RE: setting up the directory structure

[dtraughber]

Am not sure where to post this question.

I'm in process of updating my file system and directories, and am trying to follow suggestions in the new DAM Book. On page 137, there is information about putting photo files on the root directory. This is due to a different set of backup needs.

Today, during a call to with Apple Tech Support, I asked about NOT keeping these files in the User's Folder. They said it might lead to problems if these photo files were on the root directory of the drive that had the OS (but not the other drives). I am not clear on all their reasons for this. However, I feel more comfortable following the DAM Book's advice on this issue.

At present, my photo files are on a drive that does not have the OS, with backups on the other two drives (one is for buckets). The Lightroom catalog is in the User's folder>Pictures.

I'd like to move the Lightroom folder out of the User's folder, and into the Working Folder on the root directory, on the same drive as the OS. Will any move of any Lightroom catalog require re-linking every folder in Lightroom? If so, is there a sane and easy way to do the re-linking?

The 500GB drive (with the OS) will be replaced with a 1TB drive. Is there any problem making the primary drive for my photo files and Lightroom on a drive that does NOT have the OS? Is it better to do it with the OS drive?

Peter, are you putting the "Working" folder (p 136) on the drive that holds the OS?

There are four internal hard drives on my machine. The set up at present:
Drive 1 = OS (and Working Folder, which will be set up when a larger hard drive is put in)
Drive 2 = Buckets for Archive
Drive 3=  Backup for archive and derivatives on Drive 4
Drive 4 = Primary for archive and derivatives

Since I'm in the process of importing into Lightroom, rating, culling, keywording legacy images, there is nothing formally archived yet. Due to space considerations, I'll have to keep these images on the "archive" drive until they really ARE archived! Then I'll re-bucket all files when archived. Yes, all is backed up meantime.

Should I move all the unprocessed RAW files out of the archive into Working Folder at this point? The next step is to process the RAW files.

The idea is forming about how things ultimately should be system-wise, but the transition to get there is stumping me.

It helps to ask questions, as one must try to think this through when writing it down. However, I would not collapse into a false sense of security, thinking that I'm "getting" it.

Any light on what I'm missing would be extremely helpful. Getting the system set up seems to be the hardest part. (?)

Thanks,

Dadre

[danaltick]

Dadre,

I'm not an LR user so I'm not the one to tell you how to move things around in LR.  I will tell you this though.  At a minimum, keep your Working files on a separate partition from the O/S; preferably a separate internal drive for fast access.  Keep a validated mirror of that drive offline and preferably offsite.  Ultimately it's preferable to keep 3 copies, with one offline and one offsite using swappers as outlined in the book.  Setup the working files pipeline outline in the book and move your images through the pipeline and into the bucketed archive drive.  Keep the working files backed up as well as the primary archive to offsite storage.  Burn the archive to DVD/Blu-ray when your work is complete.  A DNG workflow is also highly recommended.  You should use a bucketed drive for your primary drive and another one for the offsite backup.  Peter uses a set for his originals and a set for his derivatives.  You will need to redirect LR from your Working Files folders to you primary archive as new buckets are imported.  I would also recommend reading the final chapter on migration.  It covers allot gotchas that I haven't mentioned in this short blurb.  Hope that helps.

Dan

[peterkrogh]

Dadre,
Lightroom won't lose track if you move the catalog, only the files.  So move the Lightroom catalog into the working folder.

No real reason to move the unarchived files to the working folder at this point.  Just make sure you when you do move, you do one of two things:

1. Move files with Lightroom itself or
2. Move files inside a folder hierarchy that is stable.


To illuminate point 2:
If you have 20 individual folders on drive 1, and move to drive 2, you'll need to repoint all 20 folders.
If you put all 20 folders into a single folder with Lightroom's folder panel while they still are on drive 1, then you could move the whole set to Drive 2, and only have to repoint one folder in Lightroom - the parent folder that held to 20 subfolders.
MAke sense?

As to using a working folder/working drive as opposed to keeping these files on an additional hard drive, that has a bit more to do with continuing workflow than anything else.  I set up a working drive on the workstation for all works-in-progress.  Files move through this folder on the way to the archive, which is accessed over a network.  The local drive will be faster, and less prone to errors in reading/writing, so that's why I use it.  If all your drives were connected to the same machine, then yo caould have a Working folder on the archive drive, and get the same benefit.

Peter

[danaltick]

Peter,

Yes, makes sense.  She could probably use her drive 4 as her pseudo working files until everything is bucketed to drive2.  After that, put the workflow pipeline, swappers, and 3-2-1 into place.  How's that for quick answer.  Hope it's right ;-).

Dan

[BoglePhoto]

Peter:

My question as to the working folder is a question of best practices. With a workstation, a gigabit ethernet network, a second computer/server, and a JBOD attached to the server, what is the best practice for the "working folder"?

On a laptop, I understand that the OS and the working file will probably be on the same drive, the internal HD (in my case, on a PC, my C: drive), with externals for backup on ingestion, but for my work station, which has two internal drives, isn't it better to have the OS and the other stuff (e-mails, documents, etc) on the C: drive, which is cloned, and then have a separate internal working drive for images only, set up as suggested with the folders in the book? This way that second drive (in my case the K drive) has all of the works in progress, catalogs, etc., the ingested files, etc., which is backed up to a separate swapper (other than the clone of the C: drive noted above, which is done to an external HD)? with one on site and one offsite? It makes more sense to me to have the whole drive for photos, and easier to set up the swappers to be the same size as that HD, and to mirror that HD as it is worked on.  Once the photos in working are to be put away, they get moved via validated transfer to the archive (4 HDs set up in a JBOD) which are also copied to DVD and one set of off site HD copies of JBOD.  My offsite HDs are the copies of the JBOD HDs(same size HDs as in JBOD) and the swapper of the K:

To me this makes cleaner sense of the system, as each drive has its own purpose, and as they fill up, I know what I have to move or clean out, rather than one big drive with everything on it, and more likely to fail. My second reason is that a clone of the C drive of the OS with other files balloons up very large in a big hurry using Acronis TI. I did this when I got it and filled up a 750 GB external very quickly with clones.

Bill Bogle, Jr.

[danaltick]

Bill,

Sounds like you've got an imaging workstation configuration with a separate server for the primary archive.  Peter discusses the workstation configuration on p. 243 of the new book.  That's definitely a best practice.  The only thing you might want to consider adding is an online automated (and validated) working files backup to a drive on your server.  This gives you a safety net just in case you forget about or don't have time for the swappers for few days.  Make sense?

Dan

[peterkrogh]

Yes, Bill, that's it.
Peter

[dtraughber]

Thank you, Dan and Peter, for your quick replies and advice.

Dan, thank you for pointing me to the migration chapter in the DAM Book. I have only had this new book a few days, and had not yet gotten to that chapter.

I'm working on putting down a new system on paper; additional considerations appear while reading the book and the forum and your replies. So, first things first: there's a lot of information to sort out.

I'm replacing three drives with new internal 1TBs. One is for the OS. I'm not sure what would make good use of the extra space on that particular drive. Dan recommended partitioning it, and for speed considerations, to keep the working files on a separate drive. The photos I'm working with right now are about 750 GB (but will be less after culling), so that's not a lot of growing space; there are too many photos to put on the drive with the OS. Any suggestions for this drive that shares the OS?

In any event, at present, nothing is set in stone, or down on paper, but it's getting there.

Peter, I have been reading and working with Lightroom2 more and have arrived at a better understanding in regard to my questions. I now know the consequences of moving folders with the Finder vs. within Lightroom.

In the meantime, I do have swappers and have been swapping and storing my backups (ext HDs and DVDs) off-site for some time.

Thanks again for your help.

Dadre

[danaltick]

Dadre,

I actually use a fairly small drive for my O/S.  To date, it's never even reached 30GB.  All I keep on that drive is the O/S, program files, and user folders.  I can clone the entire drive in under 6 minutes.

Dan

[Ken]

Dadre,

I actually use a fairly small drive for my O/S.  To date, it's never even reached 30GB.  All I keep on that drive is the O/S, program files, and user folders.  I can clone the entire drive in under 6 minutes.

Dan

Dan,

I do not want to stray OT, but can you remind us of what program you are using for cloning?

--Ken

[dtraughber]

Hi Dan,

Well, now there's a thought. The OS, program files, and user data is on a 500 GB drive; that's way more than enough room for that purpose.  I reckon I could replace that 1TB Seagate (the main photo drive) with the new 1TB internal, and then use the Seagate I pull out as a backup drive (that's a pretty new drive itself).  I hadn't thought of the time consideration of cloning a 1TB vs 500 GB drive. I had been thinking that I could use the extra space on a 1TB main drive for a little extra backup. Hmmm...  It does take longer for a 1TB drive to spin up.

Thanks,

Dadre

[danaltick]

Dadre,

It's not really the speed improvements (if any), as much as it is the cost savings.  Ideally it's best to have your O/S, program files, and user folders on a drive by themselves that you can easily clone for backup (see chapter 6).  The amount of time it takes to clone is not dependent on the size of the drive, but on the size of the data on the drive.  That is all that gets cloned along with the volume header information.  Ideally you should also be backing up to external or network drives.  Remember, disk space is cheap.  I personally would get a 160GB Western Digital or Seagate drive for my O/S; even better, if you can afford it, an 80GB solid state SATA drive, that would really kick your system into high gear.  I plan to build a new high end system this fall (I will post about it here), where I will be using solid state drives for O/S, PS scratch disks, and/or Working files.

Dan

[danaltick]

Ken,

I'm using Acronis True Image Home version under Windows.

Dan

[Ken]

Ken,

I'm using Acronis True Image Home version under Windows.

Dan

Thanks, Dan.  That's what I thought based on some previous posts.  I need to clone a few machines in the next few weeks before I migrate some data and programs, and I had originally considered Acronis, but there have been mixed reviews of the newer versions.  I may still consider it, but I am also looking at Macrium Reflect.  Clone software is one area where reliability is totally essential.  I remember reading a post about somebody using Mozy and they had no trouble backing up their data.  Its just that Mozy was totally unable to resotre it.  The fact that you can clone and restore quickly gives me encouragement that I can do it successfully.

Thanks,

--Ken

[danaltick]

Ken,

I use Acronis strictly for creating compressed full backup images for which it does well.  I use a different backup utility for my incrementals that runs continuously in the background with low priority.

Dan