The DAM Forum
Welcome, Guest. Please login or register.
April 24, 2014, 10:10:55 AM

Login with username, password and session length
Search:     Advanced search
28003 Posts in 5130 Topics by 2912 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Backup Strategies and Tools
| | |-+  Backups and Buckets
« previous next »
Pages: [1] Print
Author Topic: Backups and Buckets  (Read 3293 times)
drmrbrewer
Newbie
*
Posts: 26


View Profile
« on: February 02, 2006, 06:15:48 AM »

(This thread is a migration of this one: http://thedambook.com/smf/index.php?topic=281.msg1618#msg1618)

...This still leaves an issue unclear in my mind re backups and buckets...

1) Presumably you have data other than images which are important to you.  Would you use a DVD-sized bucket system for ALL your own data?
2) Say you are implementing a backup regime for a multiple user system for a medium or large corporation with terabytes of user data to protect.  Would you impose a DVD-sized bucket system on ALL your users for ALL their data?

If the answer to (1) or (2) is "no", how would you implement a backup regime to mirror only certain parts of the system?  Would you mirror all the user data in that case?  Or use an incremental backup?  Or what?

If the answer to (1) or (2) is "no", why not?

(Can I just check: by incremental I mean that the backup software only actually backs up what has changed since the last backup; I presume it means the same to you?)

Mike
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #1 on: February 02, 2006, 07:05:21 AM »

Mike,
as to 1, no, I don't use buckets for anything other than media (stills, DV, Audio).  For me, other kinds of files don't lend themselves to this kind of strategy.  I use multiple hard drive backups, with occasional DVD backups of some kinds of files

2. A medium or large corporation has a very different set of problems and resources than an independent photographer.  For one, they have full-time IT, or a contract for IT. Additionally, since so many people might "touch" the data, archiving software - rather than mirroring software - would be the right choice.  This is generally done on tape.

However, I have advised corporate entities to use my system if there is a Creative department inside a corporation that has our needs.  If a corporation has a network that is only suitable for text-size archiving, storing photos could overwhelm the system.  Sometimes the best answer is to run a local photo archive in the creative department that does not really interface with the rest of the network.

I'm sure Dan can elaborate.
Peter
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #2 on: February 02, 2006, 11:01:51 AM »

Peter,

I wrote a response to this offline, not realizing you had already responded.  I will just copy and paste it in and you can compare.  I think it correlates pretty well with your answer.  Here it is:

Mike,

First, let me say, I’m not an IT person.  You couldn’t pay me enough to do that Smiley.

So far, all of my so-called IT work has been done on my personal LAN; however, I’m sure that some of my strategies could be applied on a larger scale.

I don’t see DAM having a place in the corporate market, unless your business deals with digital assets (i.e. images, video, music, etc.).  I haven’t really studied DAM beyond Peter’s book, so I don’t consider myself an expert on the topic.  I consider the bucket system a means of implementing DAM (i.e. managing your digital assets).  That’s all I use it for.  I keep my music and videos in their own buckets separate from my images; however, I do keep all the buckets on the same hard drive, but that’s only to lower my costs.  The buckets are still numbered separately and managed separately with their own backup jobs, and can easily be moved to separate disks in the future.

I use uncompressed mirror jobs to backup my buckets for the following reasons:
•   Makes the backup disks hot-swappable with the primary
•   Saves disk space
•   Very fast
•   Allows for unlimited growth

My buckets are kept together on their own hard drives separate from my O/S, applications, databases, documents, catalogs, etc.  Currently, because I don’t have any large databases, all this information resides on the same hard drive with my O/S and gets backed up along with it as a compressed incremental backup allowing me to go back two weeks in time.  Because this data is dynamic and susceptible to viruses, worms, Trojans, etc., it’s important to keep a history of its changes; hence, incremental.  This data is also mission-critical so you should image it using imaging software (i.e. Norton Ghost, Acronis True Image, etc.) on some form of secondary storage, preferably an external hard drive.  Peter discusses this in his book.  I would recommend doing this at least monthly, and keeping these images as far back as your offline storage will allow.  

Here’s a tip if you use a LAN with more than two PC’s: use switches, not hubs.  This allows multiple backups to opposing PC’s to run simultaneously, utilizing the full bandwidth of the LAN.  I currently have 4 PC’s on my LAN; therefore, 1 can backup to 2 at the same time 3 is backing up to 4.  Once it’s setup, it’s hands off.  I only need to resort to my imaged backups if I lose my incrementals.  The incrementals provide me that extra ability to restore my O/S and apps up to the current date.  And because they’re online, it’s quick and easy.  If you don’t have a LAN, you can do your incrementals to another internal hard drive or partition.  Because they are compressed, they’re fairly small.  I use my incrementals all the time not only to restore my O/S, but also to restore applications that may have become corrupt due to application aborts or hang-ups (e.g. iView catalogs, email, Photoshop settings, etc.)  It’s quick and easy and allows me to disable the O/S built-in system-restore capability, which can incur substantial overhead and disk space on your system, not to mention its un-reliability.  My incrementals run in the middle of the night when the PC’s are not being used.

Getting back to DAM, I realize that digital assets are also dynamic to some degree (e.g. ACR adjustments to DNG’s, metalogging, derivative master files, etc.), but the nature of their dynamics I feel is minimal enough that a simple mirror backup without a history is sufficient given all the benefits listed above.  I believe Peter uses mirror jobs as well, so he must feel the same way.  Peter, feel free to comment here (or anywhere for that matter).

Mike, I hope this answers most of your questions, and hopefully it will provide a little better insight to others as well.

Best Regards,
Dan
Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
drmrbrewer
Newbie
*
Posts: 26


View Profile
« Reply #3 on: February 02, 2006, 12:57:19 PM »

Excellent information, Dan.  Thanks for the insights.  

Do you only mirror your buckets without a separate backup?  Do you mirror all your buckets?  If not, what do you do with those you don't mirror?  To show my ignorance, do you mirror with software or hardware?  If the former, what do you use?  What software do you use for general backup (GBM I guess)?

To show my ignorance again, what exactly is "imaging" (using Norton Ghost or Acronis True Image) if its different to "mirroring"?

Many thanks, and sorry for the Spanish Inquisition!

Mike
« Last Edit: February 02, 2006, 12:59:24 PM by drmrbrewer » Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #4 on: February 02, 2006, 04:58:27 PM »

Mike,

I mirror my current bucket only.  Once I move to a new bucket I add a new mirror job for it and disable the scheduling of the earlier bucket job.  Those earlier buckets remain mirrored on the backup drive, they just don't get re-sync'd unless I manually run the backup jobs; which I may do if I make a lot changes to an older bucket(s).

I use GBM for both my mirror jobs and incremental jobs.  I would suggest reading this thread for more info on the backup utilities and imaging software http://thedambook.com/smf/index.php?topic=75.0.

Dan
Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
drmrbrewer
Newbie
*
Posts: 26


View Profile
« Reply #5 on: February 03, 2006, 02:40:23 AM »

You only mirror the current bucket?  About 4.5 gigs worth?  Presumably you'd be flexible with this and perhaps mirror more if you were working on a big project with more than 4.5 gigs worth of data?

At what time points do you backup your current bucket(s), separate to the mirror job?  E.g. as part of an incremental backup.  Because, with mirroring, presumably if you accidentally delete a file before running a mirror job then that file is lost on both the primary and the mirror?

GBM looks really quite good.  I think I'll give it a demo.  I also like their summary of the the various different types of backup jobs here.

I note from this comparison table that GBM allows you to to do incremental / differential backups to DVD.  How exactly would this work?  Does GBM keep a central record of what files it has already backed up to DVD, and only backup those files that have changed, to a new DVD?  How would you restore from DVDs in this way, obviously only picking up the most recent file from perhaps multiple backed-up versions on different DVDs?  This may be something I'll have to experiment with Smiley

Mike
Logged
danaltick
Hero Member
*****
Posts: 1616

evaa-xdtb@spamex.com danaltick
View Profile WWW Email
« Reply #6 on: February 03, 2006, 08:24:17 AM »

Mike,

Yes I would be flexible there.  Remember, I can always mirror older buckets manually or automate them again for a huge temporary job.

Mirror jobs are the only jobs I use for my buckets.  If I accidentally delete a file, first it doesn't get mirrored until the following morning at 6:30am, second it will reside in the recycle bin, third it is protected by Norton File Protection, fourth it resides in my Image Working Files and is mirrored to a third PC, fifth it resides on my offline Epson P-2000, and sixth it will get burned to a DVD when the current bucket fills.  See the book and my migration thread for more info.

GBM keeps it's own local catalog just like iView.  DVD's would work the same way as hard drives for incrementals and differentals.

Dan



Logged

WindowsXP, ImageIngester Pro, RapidFixer, IVMP 3, ACR4, Photoshop CS4, Controlled Keyword Catalog, Canon EOS50D
drmrbrewer
Newbie
*
Posts: 26


View Profile
« Reply #7 on: February 03, 2006, 12:51:34 PM »

Hi Dan,

Thanks for the info.  I just installed the trial of GBM, and it sure looks good...

DVD's would work the same way as hard drives for incrementals and differentals.

... and it has an option for incremental backup to DVDs --- that's something that my current backup software just doesn't allow you to do.  It's given me an idea... one that you would cringe at, I reckon... probably one with downsides all over the place from a purist's point of view... but I do like the thought of an incremental backup job to DVD... may ponder that one for a while...

Mike
Logged
drmrbrewer
Newbie
*
Posts: 26


View Profile
« Reply #8 on: February 06, 2006, 02:46:42 AM »

The parenthesised part of this post in another thread is relevant as a follow-up here.

Mike
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!