Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
Recommendations for Hard Drive only Archive and Backup
The DAM Forum
Welcome, Guest. Please login or register.
October 23, 2020, 01:24:31 AM

Login with username, password and session length
Search:     Advanced search
28033 Posts in 5147 Topics by 2903 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Backup Strategies and Tools
| | |-+  Recommendations for Hard Drive only Archive and Backup
« previous next »
Pages: [1] Print
Author Topic: Recommendations for Hard Drive only Archive and Backup  (Read 7056 times)
Posts: 13

View Profile
« on: September 14, 2010, 08:14:41 AM »


I've been doing 23GB buckets and creating 2 HD copies and 1 BD-R for my images for the last couple years.  I'm considering of doing 3 or more HD copies and no optical discs.  I know the optical discs are a great idea, but its just been taking too much time in my workflow between creating small buckets, burning blu rays, labeling, storing, etc.

What's a good no optical disc strategy? 

I'm thinking of a few options:

1.  3 HD copies (2 on site, and 1 stored off site)

2.  4 HD copies (2 on site, and 2 stored off site) With 2 discs off site i can rotate one every other week to keep hem current, similar to working file swapper disks.

3.  3 HD copies (2 on site, 1 off site) + as needed optical discs for high value projects (maybe only 10% of work) i know i should consider everything as "high value" but some things are more valuable to me.

In any case, sounds like i will need some type of validation software.

What do you all think?



David C. Buchan
Quantum Gardener
Jr. Member
Posts: 66

View Profile WWW Email
« Reply #1 on: September 14, 2010, 04:29:02 PM »

Hi Jordan,

It's been a long time since I've been on these forums but coincidentally you have exactly the same question I've got so let's see if we can work out a solution together.

For me it is also a case of time and money. For the sake of easy maths, 1TB takes approx 200 DVDs (no blu-ray for me as yet) at a cost of AUD$200, plus hours and hours of time. Though DVD backups have saved me once I can't justify the time and cost when I can purchase 2 x 1TB drives for the same amount, automate the process and have it run much faster.

I'm thinking of:

1.  2 x onsite copies, 1 x offsite as my main backups
2.  4 x offsite archive copies at 6 monthly intervals.

At the moment I use the free version of SyncBack on Windows which works fine. Genie Backup and Sync Tool weren't at all reliable if I actually checked what was copied.

For validation I've cut my own version of SHA1 digests. I'm using the commandline version of ExactCopy to create a hash digest for each directory. Validation steps are:

1. Create hash file for each directory
2. Sync to backup drive
3. Validate hashes

Sad but true, not all files copy bit for bit perfectly as the validation proves.

Now the problem with the validation tools such as ExactCopy is they can recurse directories but to be sure you have to recreate for each folder. That takes hours and should it fail you have to start again. I've created a couple of Windows Powershell scripts that get around this problem. ExactCopy is great because it uses each CPU core which makes it as fast as possible. I've wrapped scripts around that to only call it if needed. First time around is slow but then I only create the hash file for a directory if it is missing, is not the youngest file in the directory, or if the directory has been modified in some other way. Same for validation. If I've validated I create a small marker file otherwise I know I've done this dir and move on.

It's working well and I will get to posting the scripts on my blog soon - need to find the time to document. Because, with this system, you can backup and validate cheaply (time and $) I'll be moving away from DVD copies.


Posts: 27

View Profile
« Reply #2 on: September 15, 2010, 03:25:06 PM »

Clever system, David.  Looking forward to examining your scripts.  Thanks for the ideas.
Posts: 28

View Profile
« Reply #3 on: November 09, 2010, 10:29:42 AM »

Personally I would still consider an initial backup of all images to optical as well as backing up your "high value" images this way. It is all too easy to accidentally delete something and not find out for months or years that you did so. Having that initial optical backup gives you one more chance at recovery.

Full Member
Posts: 208

View Profile
« Reply #4 on: February 13, 2011, 01:51:46 AM »

Over years, it's hard to make sure that nothing changes on a hard drive and if something changes then the validation hash becomes difficult to use.  It's so much less brain damage to use optical--but I understand where you're coming from which is why I haven't done it either.  It seems optical capacity is always a step behind.  If blu ray media were 100GB it would be an easy decision.
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!