Amazon diagnosis

Jay Ashworth jra at
Fri May 6 14:42:13 UTC 2011

----- Original Message -----
> From: "Ryan Malayter" <malayter at>

> On May 5, 3:51 pm, Jay Ashworth <j... at> wrote:
> > ----- Original Message -----
> > > From: "Ryan Malayter" <malay... at>
> > > I like to bag on my developers for not knowing anything about the
> > > infrastructure, but sometimes you just can't do it right because
> > > of
> > > physics. Or you can't do it right without writing your own OS,
> > > networking stacks, file systems, etc., which means it is
> > > essentially
> > > "impossible" in the real world.
> >
> > "Physics"?
> >
> > Isn't that an entirely inadequate substitute for "desire"?
> Not really. For some applications, it is physics:

You misinterpreted me.  I was making fun of people who think "I want
it, and therefore it WILL be so" trumps physics, of whom there are
altogether too many in positions of power these days to suit me.

> I don't have inside knowledge, but I suspect this is why Wall Street
> firms have DR sites across the river in New Jersey, rather than
> somewhere "safer".

You don't need inside knowledge; that issue's the subject of much
general press lately; that's exactly why they do it.

And they think it's good enough.

I truly wish that their finding out it's not wouldn't be so massively
disrupting for the rest of us poor slobs...
> Amazon's EBS service is network-based block storage, with semantics
> similar to the financial account scenario: data writes to the volume
> must happen in-order at all replicas. Which is why EBS volumes cannot
> have a replica a great distance away from the primary. So any
> application which used the EBS abstraction for keeping consistent
> state were screwed during this Amazon outage. The fact that Amazon's
> availability zones were not, in fact, very isolated from each other
> for this particular failure scenario compounded the problem.

Oh, so maybe "letting someone else do the cloud for you"'s a bad idea?

Whod'a thunk *that*?  :-)

-- jra

More information about the NANOG mailing list