The DAM Forum
Welcome, Guest. Please login or register.
October 25, 2014, 09:46:24 AM

Login with username, password and session length
Search:     Advanced search
28020 Posts in 5140 Topics by 2910 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  DAM Stuff
| |-+  Backup Strategies and Tools
| | |-+  Amazon S3: truly offsite, multi backps
« previous next »
Pages: [1] 2 Print
Author Topic: Amazon S3: truly offsite, multi backps  (Read 6934 times)
Steve Darden
Newbie
*
Posts: 6


View Profile
« on: October 19, 2006, 04:12:31 AM »

Walter Mossberg was jus recommending Snugmug as photosharing webservice. I wrote to Walt re remind him that Smugmug now rodes on Amazon s3 to suppllement there infrastrcture growth -pluss adding th multi-copy sites of 4 diferernt servers.

Just in case you've not yet tested any of the webDAV interfaces to S3, I'm impressed with my initial tests of Jungle Disk [no affiliation, and have only tested on OS X so far].

No stopwatch, but I'll claim less than 10 minutes to sign up for S3, download Jungle Disk, get my two S3 encryption keys, configure, archive first folder of images to S3.

    <http://www.jungledisk.com/>

The benefits of S3-type web services to enterprise are obvious. And with sufficiently low friction, I think there will be appeal to the skinny end of the long tail. These are early days yet- don't make S3 your only backup. Myself, I'm going to test S3 for the offsite archival function
Logged
David Anderson
Sr. Member
****
Posts: 287


View Profile
« Reply #1 on: October 19, 2006, 04:59:08 AM »

Seekerblog,
Is it possible to have an English translation of your post?  Huh

David
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #2 on: October 19, 2006, 05:08:05 AM »

David,
You're a funny guy...
Peter
« Last Edit: October 20, 2006, 03:43:27 PM by peterkrogh » Logged
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« Reply #3 on: October 19, 2006, 07:38:14 AM »

About S3 and JungleDisk: I keep a permanent archive of my most important images there. It's very cheap (for storage--access is expensive) and it provides a backup that's immune to any natural disasters that could affect my primary, secondary, and tertiary backups, or to media deterioration. The only problem (which is why it's so far down the list) is that JungleDisk stores data in a proprietary format, so if JungleDisk goes away, so does my data. (There's no such thing as a native S3 hierarchical file system, so JungleDisk imposes its own.)

To summarize: I wouldn't at this point use S3/JungleDisk to replace any backup that you now have, but I would consider addng it as yet another level of protection.

One more thing: It took DAYS to move all that image data to S3, but JungleDisk has an excellent interface for background uploads.

--Marc
Logged

Steve Darden
Newbie
*
Posts: 6


View Profile
« Reply #4 on: October 19, 2006, 05:44:34 PM »

About S3 and JungleDisk: I keep a permanent archive of my most important images there.

Marc,

Thanks for the based-on-experience comments. E.g., I hadn't picked up on the proprietary format issue - that's a very big issue if it is unique to JungleDisk. I thought it was standard S3 encryption - shows how little I know.

As you said, I thought the value of S3 could be disaster-proof backup of one's high-value images. Perhaps 3* and up.

Steve Darden
Logged
Steve Darden
Newbie
*
Posts: 6


View Profile
« Reply #5 on: October 19, 2006, 05:46:12 PM »

Seekerblog,
Is it possible to have an English translation of your post?  Huh
David,

Sorry about that -- way too much jargon! I wrote that late and in too big a hurry. I'll try to backfill what may not be widely known here.

1) First a caveat -- I've done no reliability testing of the JungleDisk interface to S3. I have also studied the JungleDisk forums since my intitial test - some users are experiencing glitches with large files, with Retrospect incrementals, etc. Personally, I'm not going to jump into this anytime soon.

2) Amazon's new web services are industrial strength -- Google "amazon s3", you'll find probably more than you want to know. SmugMug's Don MacAskill announced their decision to go with S3 here:

http://blogs.smugmug.com/onethumb/2006/08/12/amazon-s3-the-holy-grail/

And here is arstechnical on the Amazon announcement:

http://arstechnica.com/news.ars/post/20060314-6378.html

That SmugMug made this move, having already developed their own infrastructure good for 300 terrabytes and some 500 million images implies to me that S3 is highly cost effective for their business model. And the safety, security of S3 is very difficult for us ordinary folk to achieve. MacAskill:

   •   Your priceless photos are stored in multiple datacenters, in multiple states, and at multiple companies. They're orders of magnitude more safe and secure.
   •   We'd already built a custom, low-cost commodity-hardware redundant scalable storage infrastructure. Nonetheless, it's significantly cheaper to use S3 than using our own - especially when you factor multiple states & datacenters into the equation.


3) The challenge for non-developer users [most members of this forum] is that S3 is an API, meaning there is no external user interface. What excited my about  the WebDAV interface option is that the service is already in Mac OS X -- that is what .mac uses.

4) There are many options other than JungleDisk, and no doubt there will be LOTS of competition. Here is Jeremy Zawodny on some S3 backup tools:

http://jeremy.zawodny.com/blog/archives/007641.html

5) FYI, Mossberg's recommendation on SmugMug was Oct 19th here:

http://online.wsj.com/article/mossberg_mailbox.html?mod=djemptech

Steve Darden
Logged
David Anderson
Sr. Member
****
Posts: 287


View Profile
« Reply #6 on: October 20, 2006, 03:08:34 AM »

Steve,
Thanks for trying to explain what was, for me, an almost totally incomprehensible post. Firstly, you blithely assumed that all your readers were already familiar with Walter Mossberg, Smugmug, Amazon S3, WebDAV and Jungle Disk. This was a rather wild assumption, especially for all the non-Americans on this forum (I'm a Brit). Secondly, some of your sentences posed a major challenge for my aging brain. Let me quote two examples.

I wrote to Walt re remind him that Smugmug now rodes on Amazon s3 to suppllement there infrastrcture growth -pluss adding th multi-copy sites of 4 diferernt servers.
Even after mentally correcting the huge number of typos in this short sentence, I still only half grasp what you are trying to say.

And with sufficiently low friction, I think there will be appeal to the skinny end of the long tail.
This sentence means nothing to me. Absolutely nothing at all. Perhaps it is understandable to Americans, but on this side of the pond it is just gibberish. Please clarify. I am truly fascinated to learn what you were actually trying to say. I look forward to the opportunity to drop this phrase into my everyday conversation ...  Grin

David
Logged
johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #7 on: October 20, 2006, 03:39:20 AM »

David

An amusing read is Shoot the Puppy: A Survival Guide to the Curious Jargon of Modern Life. As in "sometimes in business, you've just got to shoot the puppy...." - just in case the phrase comes up on Dragons' Den (for Americans, that's a BBC programme).

In my moderating role I almost deleted the original post as spam, but after reading an organisational psychology PhD thesis this year, I guess my gibberish tolerance level is pretty high (or still numb). But welcome, Steve!

John
Logged
David Anderson
Sr. Member
****
Posts: 287


View Profile
« Reply #8 on: October 20, 2006, 03:56:31 AM »

John,
Love it! Despite attending many management training courses during a long career at IBM, I don't remember ever being tempted to open the kimono or kick dead whales up the beach. Nor was I noticeably touched by re-enchantment. Perhaps I wasn't paying attention.

Did you notice the 'Customers interested in this title may also be interested in:' link that Amazon provided?

"Free Advice on Caring for Your Puppy - All About Pets is the national pet care advice and information service, providing expert pet advice and support for pet owners".  Grin

David
Logged
johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #9 on: October 20, 2006, 04:07:11 AM »

David

Sometimes Amazon are too clever for their own good.

This book also tells you about a "chaviot", the "ketchup bottle effect", "herding cats", "muppet shuffle", "who moved my cheese" etc. It's a good laugh - I gave it to my friend who did the thesis....

John
Logged
Steve Darden
Newbie
*
Posts: 6


View Profile
« Reply #10 on: October 20, 2006, 03:46:29 PM »

...some of your sentences posed a major challenge for my aging brain. Let me quote two examples.
David,

You are too polite - that sentence is a good simulation of some of the spam that I receive! It is so garbled that it makes me wonder if there was a problem with the wireless transmission.  So I won't requote, but I will try to answer your questions.

Walter Mossberg writes the Personal Technology column for the Wall Street Journal. He is arguably one of the most objective, platform-agnostic commentators on computers and technology in general.

http://ptech.wsj.com/ptech.html

No doubt you are familiar with the SmugMug service - some professionals use SmugMug for stock photo sales. E.g., Andy Williams hosts his elegant website at SmugMug:

http://www.moonriverphotography.com/

As to the Amazon S3 web services, I was trying to say that the benefits for a fast-growing web company like SmugMug are obvious. And it's not just lower capital costs - there are issues such as serving peak seasonal demands. And there are the fairly obvious marketing advantages. E.g., a potential client would give SmugMug the highest marks for disaster-proof storage of irreplaceable images.

The "Long Tail": the definitive reference is probably Chris Anderson's new book of the same title:

http://www.amazon.com/Long-Tail-Future-Business-Selling/dp/1401302378

For a quick overview, Wikipedia has a useful entry. The "skinny end of the long tail" is to the right of the graph shown - representing the less popular instances [say art-house movies vs. blockbusters at the left].

My point about "low friction" is that a service like Amazon S3 could appeal to small startups who otherwise would have to devote, say, 90% of their focus to just building web-serving infrastructure -- effort which has no unique value-added. And, once easy-to-use interfaces such as JungleDisk are proven robust, even bit-players like myself may find the service valuable.

BTW, Anderson is the editor of Wired Magazine. And he has a blog on the "long tail" here:

http://www.thelongtail.com/

If you are interested, there are a couple of excellent podcasts of lectures by Chris Anderson and by Jeff Bezos [Amazon CEO] on these topics. Let me know and I'll search them up for you.

Steve
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #11 on: October 20, 2006, 03:51:53 PM »

Steve,
Thanks for providing more information.  You presence is welcome here, although I suggest setting your web browser to flag typos as you create posts. ;-)
Interesting discussion about jargon and idioms...
Peter
Logged
David Anderson
Sr. Member
****
Posts: 287


View Profile
« Reply #12 on: October 21, 2006, 03:39:18 AM »

Steve,
I read your latest post from begining to end. As far as I could see, it was totally free of typos, fractured syntax, unexplained jargon and impenetrable mixed metaphors. I understood it! Evidence is emerging that there are there may be two Steve Dardens stalking this forum ...

Despite my relief that your feet do reside on Planet Earth, I'm still going to take you to task for continuing to assume that if something is well-known in America then it is well-known throughout the world, or implying that 'best in America' is synonymous with 'best in the world'. This is not an anti-American rant, just a gentle reminder that Americans account for less than 5% of the world's population.  

Walter Mossberg writes the Personal Technology column for the Wall Street Journal. He is arguably one of the most objective, platform-agnostic commentators on computers and technology in general.
That's a rather sweeping comment on Mr Mossberg's global status, especially as you were knowingly addressing your comments at a non-American. As a Brit, I read the Wall Street Journal probably about as often as you read The Daily Telegraph, The Times, The Sunday Times, The Financial Times, The Guardian or The Independent (these are some of the quality newspapers distributed nationally within the UK). There are technology commentators throughout the English-speaking world and the fact is that very few of them are known beyond their own country's borders, whatever their merits.

Incidentally, when I checked your Mossberg link (which worked yesterday but not today) I noted that one of Mr Mossberg's primary objectives was to write about technology in plain English ...

No doubt you are familiar with the SmugMug service - some professionals use SmugMug for stock photo sales. E.g., Andy Williams hosts his elegant website at SmugMug
I'm not quite sure why you are so certain a Brit would have heard of SmugMug (or that Andy Williams). Many online services of American origin are indeed known worldwide, such as Amazon and eBay, but it would be rash to assume that they all have a global presence. Perhaps SmugMug is better known to Mac users outside the States (I use a PC), but, for whatever reason, I've never heard of it before.

BTW, Anderson is the editor of Wired Magazine. And he has a blog on the "long tail" here:
http://www.thelongtail.com/
Again, you just assume that Wired Magazine means something to a non-American. It didn't to me, though a UK web search gave me the following link: http://www.wiredmag.co.uk/. Not, I suspect, the same Wired Magazine as sold in the USA  Smiley.

Thanks for the explanation of the 'long tail' reference. Sadly, I fear that I may never have a realistic opportunity to use the phrase "with sufficiently low friction, I think there will be appeal to the skinny end of the long tail" - unless I meet some visitors from Planet Gobbledegook Smiley. Anyway, despite everything, welcome to this excellent international* forum! You clearly are a technically knowledgeable person with much to offer.

David

* The forum owner and senior guru is Peter Krogh (American) while John Beardsworth (British) is 'Global Moderator' - a post that is presumably equal to that of Kofi Annan.
« Last Edit: October 21, 2006, 05:12:59 AM by David Anderson » Logged
mikeseb
Jr. Member
**
Posts: 91


View Profile WWW
« Reply #13 on: October 21, 2006, 07:04:35 AM »

John Beardsworth (British) is 'Global Moderator' - a post that is presumably equal to that of Kofi Annan

An insult to john's effectiveness as forum moderator....
Logged

David Anderson
Sr. Member
****
Posts: 287


View Profile
« Reply #14 on: October 21, 2006, 07:15:44 AM »

Tut, tut, Mike. I meant no ill will to either John B or Kofi Annan. I merely wished to highlight the exalted nature of John's role.

David

Logged
Pages: [1] 2 Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!