Category Archives: Data Storage

World Backup Day

Once again, it’s World Backup Day! While it’s not as fun as Talk Like a Pirate Day, it’s arguably more important. All of us have important digital stuff that we’d hate to lose. So if the lack of a solid backup plan is something that’s bothering you (even a little), take the opportunity to do something about it.  Here are some suggestions.

Send in the Clones
If all your stuff can fit on one single hard drive, then you’re in luck. You can make a clone of your drive.  A clone is simply a copy of the drive, written out to another hard drive. It’s really useful if your hard drive crashes. And a clone that lives in a separate place from your laptop will give you protection in the event of loss, damage or theft of the computer.

Clones are easy to make, and offer a high level of protection (as long as you update them regularly). I think of a clone as a disaster-recovery backup. As someone who really values my data, I like to keep an extra clone stored offsite, in case there is a fire or theft that destroys both my laptop and my main clone.

You can read about making a clone over at dpBestflow.org.

Krogh_150331_WD_Air

I’ve been using this nice little WD My Passport Air for my clone, it’s small, light and durable. It also has built-in encryption so your stuff is protected even if the drive is lost. 

Online backup
While I think everyone needs a clone for fast recovery, I’m also a big fan of Backblaze for continuous off-site backup. It’s a real set-it-and-forget-it system. It costs $50/year per computer to make a duplicate of your entire computer up to the cloud. This protects against the threat of total loss of onsite data, as well as any files that have not been backed up to offsite storage.

Backblaze is particularly valuable for family members or other who are not vigilant about backing up their stuff. I set up both my daughters before they went off to college, and, wouldn’t you know it, one of them dumped a pitcher of water on the keyboard of her laptop during freshman year.

Note that Backblaze is not really designed for large image libraries that many photographers have.

PhotoShelter or other web service
You can also use a photo-oriented service for backup. If you are a PhotoShelter customer and you use Lightroom, you can automatically publish images to the cloud. I have mine set to publish high quality JPEGs from all 4 and 5 star photos.

Publish Backup

 

 

 

 

 

Lightroom’s Publish Services can be used to backup images to the cloud mostly automatically. This can provide a current JPEG (or original file) backup that is updated as new files are added to the catalog. 

Big Drives
If you have a lot of data like photos and videos, you might want to get some big drives for backup. WD is now shipping 6 TB drives that are about $250. That’s a heck of a lot of data in a small package at a reasonable price.  There’s no excuse not to keep those photos backed up.

Krogh_150331_Toaster(Back them up twice if possible – once on-site, and once off-site, for a total of 3 copies.

Here’s a really economical way to backup files. Get a bare drive and a “toaster”. You don’t want to use the toaster for everyday use, but they are great for backup.

 


Don’t let Perfect be the enemy of Good

It’s easy to be overwhelmed by all the considerations that go into a perfect backup system. So don’t try to be perfect, try to be better. If you don’t have a clone, get one. If you travel a lot, then online backup may be a good addition. And if you have only onsite backup, consider adding an off-site.

Each time you make an improvement to the system, you add more protection, and reduce the chance that you’ll lose important data.

Disclosure
I’ve recently been working a bit with the folks at WD. They have sent me some equipment to evaluate, and they sponsored my last talk at PhotoPlus Expo. And a few weeks ago I went to a Product Summit in Laguna Beach. I still have to buy most of my own hard drives, and I’ll typically buy WD when I’m spending my own money.

I have also been working with PhotoShelter to create a new service for people who buy photographs. Again, I’m working with a company I really believe in, because I really believe in them.

Cloud Wars

The competition to provide you with cloud storage is starting to reach a fevered pitch. It’s now possible to add excellent cloud backup to your storage system for a very reasonable cost. Some of these costs remain artificially low, and may therefore not be reliable in the long run. But we’re also seeing the big players in computing (Google and Amazon) offering really low pricing.

Krogh_061001_8516

First, a word of caution
We’ve seen some low-cost options for years. This includes services like Carbonite and Backblaze that have claimed “unlimited” storage for prices around $50/year. This means that someone like me with a dozen terabytes of data will be a money-loser for each of these companies. I’ve always been distrustful of these plans, fearing that the companies will go the way of Digital Railroad, which shut its doors with little advance notice in 2008.

Carbonite gets around the super-user problem by limiting the cheap backup service to your internal drive. As you add external disks, the price goes up. (Let’s also take a minute to note that Carbonite does not forecast profitability anywhere on the time horizon, which is problematic.) Backblaze does allow for truly unlimited data, and explains their strategy by saying it will average out between low and high volume users. This is okay for backup, as long as you realize the service may go away someday, and it’s not your only backup.

(Note: I personally use Backblaze for my computers and for my family. I’m currently testing the unlimited storage with my own archive. You can get a discount off Backblaze by clicking my affiliate link.)

The big boys jump in
Last summer, Amazon rocked the world of online storage by offering a new cloud backup and archiving service called Amazon Glacier. The price for the service came in at 1/10th of Amazon’s regular S3 pricing. You can now store a terabyte of data in Amazon’s cloud for $10/month.  This one is a game-changer. Amazon is the 800lb gorilla in cloud service, so the prices that they set will determine what the rest of the market does.

Krogh_060729_3167

Amazon Glacier is positioned as a real backup or deep archive solution. They say it may take up to 5 hours to access the data, so it’s definitely not a place to store stuff you expect to access frequently. But it does promise great safety and reliability from a blue-chip company.

(I’ve heard, from a very good source, that Amazon can offer this service because they are making use of some “free” capacity. In order to speed up its regular service, Amazon is using the outer rings of the hard drive platters, which deliver faster data throughput. So the inner rings were sitting on drives unused. They created Glacier to make use of this spare capacity.)

Google responds
A few weeks ago, Google matched Amazon’s bet, and even raised it. Not only did they match the $10/terabyte/month price, they made the offer on Google Drive.  This means that Google is offering the price on storage that is always on, not just a backup service.

DriveOnWhile Google will probably lose money on this specific service, it’s part of a larger strategy from the tech giant.

(Note, I’ve been slogging through Google’s Terms of Service to get an idea of exactly what rights you give to Google Drive, and it’s not totally clear to me. It does look like private data stored on Drive is private. But other stuff, like your public photos on Google+ do seem to give Google a  non-terminable license to republish.)

It’s really about “My Stuff Everywhere”
The real competition at work here is not about collecting money for storage. The real competition here is to become the universal shared storage system which can work across all your devices.

Dropbox has been the category killer for this service, seamlessly  sharing between you, your friends and coworkers, your computer(s) and your phone. It has been able to do this where Apple (and others) have failed numerous times. Dropbox has rocketed up in value, and is poised to become even more valuable.

The companies that become successful in creating a shared filesystem  are well-positioned for long-term success.  This kind of engagement is hard to pull away from, since  you build it into your collaboration and your fundamental relationship with your own media.

DAM Edition 3.0 Postcard.indd

In The DAM Book 3.0, I’ll dive into the use of cloud storage as part of a DAM strategy. This new development in pricing and strategy offers some excellent value for photographers looking for storage, backup and sharing services.

Hard Drive Reliability Study

Every hard drive is out to get you, (but some more than others).

I’ve used that as a laugh line in my lectures for a bunch of years. Whenever people ask me which drive to buy, I point out that even the best quality drive can experience a sudden failure. But you can lower your odds of a problem.

Backblaze is a company that buys drives by the truckload. And they buy the most cost-effective drives they can. They are nice enough to publish the failure statistics for specific brands and models. Last week, they released a round of these numbers, and provide some good context for them.


This graph is from an article on the Backblaze blog that outlines the failure rate of various drives they use.

The short answer is that Hitachi drives provided the best reliability for Backblaze, and Seagate was the worst. In the case of the Seagate 1.5TB drive, the numbers are really bad. I’ve been buying (and recommending) Hitachi drives for a couple years now. Good to get a little more empirical evidence.

BTW, I like Backblaze as a cloud-based backup service. I don’t use it for my own work, because I have things taken care of locally. But I’ve installed it on the computer my daughter has taken to college with her. It makes a cloud backup in the background as she adds or changes files.

Using Multiple Lightroom Catalogs

When you use multiple catalogs in Lightroom, it’s important to be clear about why you are splitting your collection and what you hope to accomplish. In my new book, I outline the most common of these reasons so you can create a purpose-driven workflow. Most people’s workflow will fall into one of the following groups:

Multiple Master Catalogs
Project and Master Catalogs
Working and Archive Catalogs
Synchronized Catalogs
Satellite Catalogs.

Each of these workflows has a dedicated chapter outlining the goals and how to achieve them. The following video helps you understand what each of these configurations includes.

 

DNG Verification in Lightroom 5

I’ve been looking forward to the day this can be announced since 2007. In Lightroom 5, there is now a one-click solution to verify an entire collection of DNG files. It’s a really simple idea, with pretty huge ramifications from a data management standpoint. Interestingly, it’s nearly absent from any Adobe marketing materials for LR 5.

Read all about it after the jump.

DNG Verification

Near the bottom of Lightroom 5’s Library menu, is an item that lets you validate an entire collection of DNG files with a single click. It’s right below the “Find Missing” command. These two tools, when used together, offer excellent verification workflow.

Continue reading DNG Verification in Lightroom 5

You’re going to need a bigger boat…

I got my D800 today, and I feel a bit like Roy Scheider in Jaws after he saw the size of the shark.  The files that come out of this camera are huge and remarkably good.  The 36 megapixels are also pushing the envelope of all of the rest of my equipment.

Lenses
The camera is so sharp that it is showing focus falloff where my D700 did not. Even great modern lenses like the 14-24 are showing signs of image imperfections that I have never seen before.  I suppose I’ll need to test all of my lenses and see which ones are up to using on this camera.  I also expect that I’ll need to test them at all apertures.

There’s a potential limits of diffraction problem with this kind of pixel density.


Lonaconing Silk Mill, Lonaconing, MD 60mm Micro Nikkor @ f/8
Click here for the Zoomify version 

Cards
I’m getting 175 images on an 8 GB CF card. That’s going to go quickly. I have a bunch of much larger SD cards (32 GB), but they are slower.  Again, it looks like I’ll be doing some testing here.  I want first to see if any of the cards will produce a slowdown in shooting speed as they struggle to keep up with the data writing. After that, I’ll want to see what download times are for the various cards.  This will certainly be important in the field.

Hard Drives
I’ll be filling these up much faster with 40-50 MB raw files (14 bit, lossless compressed). This will certainly mean new portable drives for my upcoming trip to China, as well as for any extended location shoot.  And the archive drives will also be filling up faster, so there’s another purchase there as well.

Computer
These files are big, and process slowly. I have a feeling I’m going to need as much speed as I can get. New iMac?  New Macbook Pro?  Not sure.

Web publishing technology
There’s also a need to be able to view these images over the web. Photoshop has come with something called Zoomify – linked here – that can help with this.  But there’s some new technology called Piqsure that does this with HTML5 in some pretty cool ways.  More on that soon.

Shutha.org is live

For the last couple of months, I’ve been working on a project to help African photographers put their photos and multimedia into the world marketplace. Shutha.org is a free online learning resource, geared to professionals and aspiring professionals in the Majority World.  It was funded by World Press Photo and the Dutch Postcode lottery.  The project was run by Dave and Rosanne Larsen at Africa Media Online.

There is a comprehensive set of learning resources here, including help for business basics, marketing, business practices, as well as technical information.  Dave, Rosane and Dominique LeRoux were in charge of the business, sales and marketing materials. D.J. Clark produced a great section on multimedia production, Graeme Cookson provides background on imaging technology, and I wrote about Lightroom and how to create a safe and cost-effective digital photo computer system.

The entire project will look familiar to those of you who have seen dpBestflow, since it is powered by the same Drupal software that used over there.  It’s hard for me to see how we could have created this project without the generous support of ASMP. They contributed the use of their Drupal customizations, and paid for the work by Context Solutions, the excellent development team that worked on the ASMP.org site as well as dpBestflow.org.

Here’s the first movie in the Lightroom lesson plan: it outlines some of the creative possibilities offered by Lightroom in image adjustment.  I’ll provide more background about the project, the material and the team in future posts.


This movie shows “before and after” examples of photographs adjusted in Lightroom. It is part of the extensive and free training resources on Shutha.org