The DAM Forum
Welcome, Guest. Please login or register.
September 02, 2014, 06:58:58 AM

Login with username, password and session length
Search:     Advanced search
28016 Posts in 5137 Topics by 2910 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Backup Strategies and Tools
| | |-+  SuperDuper verification
« previous next »
Pages: [1] Print
Author Topic: SuperDuper verification  (Read 8696 times)
agrarian
Newbie
*
Posts: 2


View Profile
« on: December 18, 2009, 11:33:41 AM »

I'm starting to implement some of the suggestions from The DAM Book. Right now, I'm looking at backup software. I see from other forum posts that SuperDuper is a favorite for backup software with some folks. 

On page 225 of The DAM Book, Figure 6-13 shows that Chronosync has the option to verify that a file has been copied correctly by doing a comparison of the copy with the original.  I looked at SuperDuper's web site and manual to see if it had that capability.  I found nothing.  So I contacted SuperDuper's developer, and asked about it.  Here is his reply:

"No, because modern disk controllers flag errors when invalid data is read/written... I mean, does your word processor check that what it wrote to the disk when you saved the file is the data you saved? No, because if there's an error, you'll get an error... "

I'm sure there's some truth to what he says, but I'm wondering now if SuperDuper is really qualified for DAM purposes.  It looks to me like the extra step of file verification would be better for DAM purposes.  Perhaps Chronosync wold be a better choice.

I used Carbon Copy Cloner to copy files over to a new drive, but now I'm wondering if that is a good choice.  I can't find anything in their manual about file verification, either.  After making the switch to a new drive, I used Apple's FileMerge to compare the new files with the old, and they seem to be ok.  But it would be better to have it all in one package.

Any thoughts on this would be appreciated.

Thanks,
Tim Baker
Logged
BobSmith
Full Member
***
Posts: 239

bobsmith@mac.com
View Profile WWW
« Reply #1 on: December 18, 2009, 03:38:56 PM »

All I know is that Chronosync's verification feature saved my rear in a large way recently.  I was migrating data to larger drives on a new server setup.  I copied buckets containing about 30K DNG files in one of the moves.  The process seemed to proceed just fine but Chronosync flagged about 5K of the files as not being copied correctly.  I think it was a cable issue on an external drive.  The damaged files were randomly placed among the lot.  I wouldn't have noticed the issue for ages had I not used that feature, and the bad files may well have been moved into other backups.  Once I knew of the problem I could see that each of the bad files was noticeably slightly smaller in size than the good ones... but not so much so as to be painfully obvious a first glance.  The files were all there in the proper place but they were unreadable by any of a variety of attempts to open them.  I simply re-did the entire migration with a different drive setup and all went smoothly.  I ran ImageVerifier on everything as well to be sure that I had a good archive.  I've happily used Chronosync for years but that little episode made me a huge believer in it's verification process.

Bob Smith
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #2 on: December 21, 2009, 04:23:00 PM »

Tim,
I'm surprised that SuperDuper! does not have that capability.  But if the developer does not think it's necessary.

I would say that he is simply wrong.  While the disk's firmware can validate what's written to the disk, the problem can happen at any number of places between one drive and another.  Like Bob, I've had personal experience with Chronosync's validation saving my butt (and also helping me to identify the place in my system where corruption was happening).

I personally think Chronosync is well worth the cost.
Peter

Logged
agrarian
Newbie
*
Posts: 2


View Profile
« Reply #3 on: December 21, 2009, 07:42:01 PM »

Bob and Peter... thanks for you replies.  It looks like Chronosync is the winner!

I've used Norton Ghost on the PC side for years to clone my system drives, and ALWAYS used their verification function... it's too risky without it.  But NG won't work on a MAC.  The closest thing I've found for a MAC is CopyCatX.  It is supposed to be able to clone everything, including the Bootcamp partition and MAC partition, all in one pass.  The last time I checked, Carbon Copy Cloner and SuperDuper! can't do that.  Whether CopyCatX has a verification function, I'm not sure.  I've only used it for my system drive.

When I moved my photo files to a larger disk recently, I used Carbon Copy Cloner to move the files.  Then, I downloaded Apple's Xcode and used the FileMerge utility to compare files between the original and the copy on the new drive.  Assuming I used it correctly, everything was ok.  Kind of a pain to use with so many files, but the price was right.  Maybe I should have waited and used Chronosync.  I am going through everything and making a visual inspection.  So far, all is well.

Thanks again,
Tim
Logged
Mary
Newbie
*
Posts: 29


View Profile
« Reply #4 on: January 13, 2010, 06:02:13 AM »

Here's what Dave at Shirt Pocket told me about validation and gave me permission to post.  I'd sure like someone to explain this to me.     --Mary Undecided
-----

SuperDuper! does not do a verification. Never has.

Here's why: Retrospect's verify-after-backup was generally useful in the 'old' days of Retrospect and DiskFit, back when the disk controllers were 'stupid' on old tape drives and floppies. Those controllers didn't really 'check' their own writes.

Modern controllers have checksums and ECC built into read operations and give errors when writes fail. So a 'verify' won't tell you much of anything - basically, it'll just tell you what we already tell you: a write (or read) failed.

After all, when you write a file to disk apart from a backup, do you 'verify' that write? No... if you get an error, you get an error. Otherwise, you assume the write is correct. The same goes for a backup.

If you're concerned about files failing that have already been written, I suggest investing in a RAID-mirrored backup drive, such as the LaCie 2big (set to Mirrored RAID mode). That will protect against drive failures pretty effectively.
--
Dave Nanian
Shirt Pocket
---

Logged

Using MacBook Pro, Snow Leopard, and Lightroom.
BobSmith
Full Member
***
Posts: 239

bobsmith@mac.com
View Profile WWW
« Reply #5 on: January 13, 2010, 10:46:45 AM »

Here's what Dave at Shirt Pocket told me about validation and gave me permission to post.  I'd sure like someone to explain this to me.     --Mary Undecided
After all, when you write a file to disk apart from a backup, do you 'verify' that write? No... if you get an error, you get an error. Otherwise, you assume the write is correct.

Rare... but never the less I've certainly experienced numerous instances over the years where a file written to disk without any error turned out to be corrupt when attempting to open again a short time later.

Also, in the case I was discussing earlier the transfer involved moving files over a network, not just to another local disk.  I would assume that's very common among readers of this forum.  I would guess (from personal experience) that it's at least a bit more likely to produce errors in the moved data in ways that are different from files written locally (which seems to be specifically the scenario that Dave is describing).  I can certainly imagine a case where the computer doing the writing doesn't get an error but the computer receiving the data does so without error but then doesn't fully write the data to its disk correctly. I don't know the nuts and bolts of what's happening and how it's verified but I've seen first hand cases of files being corrupt on the receiving end after a simple non verified copy that seemed to happen without any errors.  Only attempting to open the files later or using a verified system like Chronosync offers showed the error that occurred during transfer.

I use mirrored raids for some internet servers that I run.  Such a setup allows very quick disaster recovery but it opens up all sorts of other cans of worms.  Errors... human or otherwise... are immediately duped to an otherwise good disk.  It's not a substitute for backups in any way at even the simplest level unless you are frequently breaking one member of the raid off and letting the raid rebuild with a new disk inserted (easy but quite time consuming).  The removed disk is an instantly useable snapshot of where the system was at that time should the raid go belly up for whatever reason.

Note... I own and use Super Duper but not for this type of situation.  With the latest version of Chronosync being able to do bootable backups I find myself using Super Duper less and less.

Bob Smith

Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #6 on: January 19, 2010, 05:22:15 AM »

>After all, when you write a file to disk apart from a backup, do you 'verify' that write? No... if you get an error, you get an error.

Actually, I would say this is wrong. ANY time I make a critical transfer, I do a validation (I simply use Chronosync or Syncback to do the transfer). And I only started to do this because it actually bit me once.  I had a drive that appeared to function normally. Files transferred without complaint. However, I periodically found corrupted image files, such as this one:

http://www.dpbestflow.org/data-validation/data-validation-overview#did

It took me many months to find out where this was coming from - ultimately it was a bad drive enclosure. The *only* way I could have found out was Chronosync, which kept reporting random verification failures. Eventually, it dawned on me that these failures were the cause of the corruption.  removing that enclosure from my system solved the problem.

Note that the OS did *not* report any error writing to the drive.

Peter
Logged
tho3hite
Newbie
*
Posts: 9


View Profile
« Reply #7 on: March 03, 2010, 07:19:57 AM »

What about the new Retrospect 8 for Mac? I had a love-hate relationship with the old Retrospect for may years. I hated the awful interface but I loved having those backups.

Retrospect also saved my butt on many occasions.

My copy is way out of date and I want to either upgrade to ChronoSync or Retrospect 8. I don't thing SuperDuper has the features that we need for the DAM workflow.

Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #8 on: March 04, 2010, 10:03:52 PM »

Thomas,
I'd be pretty surprised if it did not have verification, but I don't know for sure.  HAve you looked at the product literature?
I do think Chronosync is a great tool for this.
Peter
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!