The DAM Forum
Welcome, Guest. Please login or register.
October 22, 2014, 04:05:47 PM

Login with username, password and session length
Search:     Advanced search
28018 Posts in 5138 Topics by 2910 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Backup Strategies and Tools
| | |-+  Backup strategies - a primer
« previous next »
Pages: [1] 2 Print
Author Topic: Backup strategies - a primer  (Read 7361 times)
Rick McCleary
Full Member
***
Posts: 240


View Profile WWW Email
« on: December 21, 2005, 08:40:21 AM »

In the past, all my backups have beeen done manually in a fairly unorganized way.  As the avalanche of digital files comes cascading down, I find that I need to be much more organized about the mechanics of the backup protocol.

Perhaps someone with a deep knowledge of the do's and don't's of backup can weigh in with a step-by-step recommendation for approaching this issue.  (Peter, Dan?)

There are a number of backup utilities available that automate the process.  I have downloaded a trial version of Synk.  It seems to do what is needed, but I'm not sure if I'm missing something important.  (I'm not a programmer or computer engineer!)

For my sake, and the sake of others who might read the advice, please consider me a novice when it comes to the world of automated backup.  I need the type of instruction Knute Rockne gave to his new football players: "Gentlemen, this is a football..."

Continuing to live and learn - Rick
« Last Edit: February 12, 2006, 06:24:49 AM by peterkrogh » Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #1 on: December 21, 2005, 09:28:58 AM »

Hi Rick,

I'm not familiar with Synk, and I'm not sure if you're running on a PC or Mac, but if Synk supports mirror jobs, I would go through the Synk wizard and get familiar with mirror jobs.  If not, and you are running on a PC, I would go here http://www.genie-soft.com and download a trial version of GBM or GBM Pro and go through the wizard and learn how to create a mirror job and place it on your desktop for single-click execution.  It's actually easier than you might think.

Here is a statement from a thread I started earlier called "A Great PC Backup Utility" in the Software Discussion area:

Quote
For media, I use separate GBM jobs that are mirror jobs over my LAN without compression....media is already compressed.  Mirror jobs lend themselve well to huge media backups because their only function is to synchronize the primary with the secondary (i.e. additions and deletions)....very fast and independent of backup size (except maybe for the backup search operation to find out what's changed).  The only drawback is no incremental capability; however, how many times do you need to go back in time with images....my current thinking is it's worth the price/risk.

If you have any more questions after doing this, feel free to post.

Hope this helps,
Dan
« Last Edit: December 21, 2005, 09:30:52 AM by danaltick » Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
AlanDunne
Full Member
***
Posts: 185


View Profile Email
« Reply #2 on: December 21, 2005, 09:39:54 AM »

Rick,

I too support the need for this primer. I am working through these very issues myself. The DAM Book has shown that I had some holes in my old approach for backup and archiving that I am trying to plug. I think the discussion will need to have a conceptual component as well as platform specific components (PC vs MAC). From what I have seen, Dan aka danaltik seems to know a lot about this topic and has the right IT background.

By the way, I think it was Vince Lombardi you are quoting, but since I am a Canadian my culteral references may be incorrect ...

Cheers ... Alan
Logged
Rick McCleary
Full Member
***
Posts: 240


View Profile WWW Email
« Reply #3 on: December 21, 2005, 10:40:12 AM »

I think the discussion will need to have a conceptual component as well as platform specific components (PC vs MAC).

Absolutely.  Information about the structural underpinnings would be very helpful prior to diving into details about specific platforms and tools. 

BTW, I'm on a Mac.

Looking forward to more lifelong learning!

Rick
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #4 on: December 22, 2005, 05:53:36 PM »

Rick & Alan,

Sorry I'm just now getting back to you on this topic, I needed to look around a little bit.  Let me refer you to this article for a few backup concepts http://www.genie-soft.com/community/columns/general_backup_concepts.html.  After reading through this, you might want to check out some of these articles as well http://www.genie-soft.com/community/columns.html.  I really like the article, "To Backup or Not to Backup"....enjoy.

Dan
Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
AlanDunne
Full Member
***
Posts: 185


View Profile Email
« Reply #5 on: December 23, 2005, 09:38:27 AM »

Dan,

thanks for sharing the link. There is some useful information there. I have been using Genie for several days and it is an excellent program.

Given that you have been studiously reading Peter's book, I would like to pick your brain on what you feel is the best backup type for various aspects of DAM. Here are my thoughts:

- "Working files" - Since these are actively being modified, use Incremental backup
- Archive - Since these are stable, use Mirror backup.

One thing I was not able to figure out is if Incremental will back up a new file file added to a folder if that folder is defined in the backup job. Also since working files are eventually moved to the archive, if the working backup is Incremental, does it keep maintaining old copies of files no longer part of that backup job? Maybe I have it backwards

Any thoughts? How are you using this tool to implement the strategies in the DAM Book?

Cheers ... Aln
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #6 on: December 23, 2005, 05:06:40 PM »

Hi Alan,

No you do not have it backwards.  Let's start from the beginning.

I actually prefer to use mirror jobs for all my media, including the WorkingFiles folder.  Here's why:

    1) I don't really ever have a need to go back to earlier versions of an image.
           a) For RAW files, it's easy enough to just edit them and undo the adjustments, or even globally set them all back to their defaults using Bridge.
           b)  For layered TIFF's, my history for the most part is maintained in the layers.
    2) By using an uncompressed mirror job, I can actually peruse the backup folders without the need to use the backup utility.  Afterall, it's just a mirror of my directory structure.
    3) Mirror jobs consume much less disk space because they don't have to hang on to the older versions.

Quote
One thing I was not able to figure out is if Incremental will back up a new file file added to a folder if that folder is defined in the backup job.
Yes.

Quote
Also since working files are eventually moved to the archive, if the working backup is Incremental, does it keep maintaining old copies of files no longer part of that backup job?
Once the files go into an incremental archive, they do not get deleted, so yes the old copies stay there until you create a new normal/full backup, purge the old backup set, and start the incremental set over again beginning with a new normal.  That is actually how I backup my O/S.  I use an incremental backup with rollback consisting of six increments and I tell GBM to do a purge and only keep the last two backup sets.  Since there are six incrementals and one normal making up a full set, if my PC is on every night, that takes seven days to complete a full set.  If I keep the last two sets, I can go back in time up to two weeks with my O/S.  For my O/S, I consider that very important because viruses, trojans, worms, etc. can go undetected for a period of time, and I don't consider this something worth worrying about with images, the price is too heavy diskspace-wise.

Quote
Any thoughts? How are you using this tool to implement the strategies in the DAM Book?
I intend to use strictly mirror jobs for DAM.  Here are the jobs I will create:
    1) WorkingFiles backup to offline device (icon on desktop)
    2) WorkingFiles backup to online storage (scheduled nightly)
    3) Primary to secondary backup to online storage for original archive (since my secondary stays online, I will schedule it nightly).
    4) Primary to secondary backup to online storage for derivative archive (also scheduled nightly).

Note - when performing backups; especially of media, it's a good idea to disable the auto-protect on your anti-virus software.  It can slow the backup to a crawl.  At least I know Norton Antivirus does that.

Hope that clears it up.

Dan
« Last Edit: December 23, 2005, 05:30:45 PM by danaltick » Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
AlanDunne
Full Member
***
Posts: 185


View Profile Email
« Reply #7 on: December 24, 2005, 05:43:20 AM »

Dan,

thanks once again. This matches the strategy I was striving for but you have provided some of the missing pieces to the puzzle.

Two follow up qustions, then I will let it go ...

1) If you run set up these backup jobs to be schedule to run during the night, how do you disable and re-enable the auto-protect in Norton Anti Virus?

2) If you have a file in one folder, then move it to another folder without changing the contents of the file, how does Genie see the change? Does it see it as one file deleted and a new file in another location, or does it track the path change and save that only?

Thanks for your contribution to this forum. It is good to see people other than Peter contributing to the success of this forum.

Cheers ... Alan
Logged
cabrackett
Jr. Member
**
Posts: 83


View Profile Email
« Reply #8 on: December 24, 2005, 07:42:14 AM »

Dan
Just sort of jumping into this discussion, I need some info re Genie. I have been running it on a Windows platform which just died and am looking to switch to a Mac platform. Genie doesn't run on Mac without emulation and who thinks emulation is a good deal for a backup program? So is retrospect the only answer? Any thoughts.

I do like Genie, but I may want a Mac more.

Chuck Brackett
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #9 on: December 24, 2005, 08:33:15 AM »

Chuck,
Why don't you start a thread called something like Backup software for Macintosh.
Peter
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #10 on: December 24, 2005, 08:42:59 AM »

Quote
1) If you set up these backup jobs to be scheduled to run during the night, how do you disable and re-enable the auto-protect in Norton Anti Virus?
I use a DOS batch script that disables the service, calls GBM, then re-enables the service.  I schedule the script.

Quote
2) If you have a file in one folder, then move it to another folder without changing the contents of the file, how does Genie see the change? Does it see it as one file deleted and a new file in another location, or does it track the path change and save that only?
It would show it deleted from one folder and added to another.  The file itself would be backed up to a new location now on your backup drive.

Quote
Genie doesn't run on Mac without emulation and who thinks emulation is a good deal for a backup program? So is retrospect the only answer? Any thoughts.
Yes, I would go with Retrospect on a Mac; however, I'm not a Mac guy, so Peter may advise differently.

Dan

Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
jeremyrh
Jr. Member
**
Posts: 52


View Profile
« Reply #11 on: March 16, 2006, 05:56:57 AM »

Yes, I would go with Retrospect on a Mac; however, I'm not a Mac guy, so Peter may advise differently.

Dan

I wouldn't. I used to use it, but after a number of expensive version upgrades I started to wonder if I really wanted to generate a big pile of archives of versions of stuff all kept in a compressed proprietary format.

Now I use SuperDuper (http://www.shirtpocket.com/) to make multiple bootable mirrors of my drives, so all my files are easily accessible in case of disaster. The price of hard disks is now low enough that this is a feasible option, and it's much the easiest.
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #12 on: March 16, 2006, 06:44:44 AM »

The archive conccept that Retrospect uses is a reasonable solution for text based documents, but I would suggest that it is inappropriate for image data.  It simply generates too much data.  The Retrospect model would want to keep a different version of the image files each time you adjust them. 

So if you bring in 10 gigs of images from a shoot, adjust them once and convert too DNG, you might end up with nearly 30 gigs of data to archive, when all I would really be wanting to save and backup is the final adjusted DNG folder.

For this reason mirroring is a better strategy for image file backups.
Peter
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #13 on: March 16, 2006, 07:33:19 AM »

Peter,

If I'm understanding you correctly, Retrospect does not support mirror backups.  If that's the case, then I certainly would not recommend it for any type of media archiving.

Dan
Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
jeremyrh
Jr. Member
**
Posts: 52


View Profile
« Reply #14 on: March 16, 2006, 07:49:19 AM »

Peter,

If I'm understanding you correctly, Retrospect does not support mirror backups.  If that's the case, then I certainly would not recommend it for any type of media archiving.

Dan
Correct. It is different, and has a different purpose. Suppose you were writing a book and wanted to have the option of going back to the version of Chapter 6 that you wrote last week, and you since made a mess of, you might use Retrospect to go back to your old back-up to get that version of your work - assuming your backup is not corrupt n some way, which is not easy to check with absolute certainty.

If your hard disk crashes, you can still recover your files once you've been out and bought a new disk, but it will be a pain.

With a mirror, or cloning, software like SuperDuper you make an exact bootable copy of your hard drive. If your hard drive flakes out, in the short term you can boot off your external drive and keep working. When you replace your bad drive you can simply clone across to it from your backup drive.

Logged
Pages: [1] 2 Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!