Amazon diagnosis

Thu May 5 17:45:10 UTC 2011

On May 1, 2:29 pm, Jeff Wheeler <j... at inconcepts.biz> wrote:

> What it really boils down to is this: if application developers are
> doing their jobs, a given service can be easy and inexpensive to
> distribute to unrelated systems/networks without a huge infrastructure
> expense.  If the developers are not, you end up spending a lot of
> money on infrastructure to make up for code, databases, and APIs which
> were not designed with this in mind.

Umm... see the CAP theorem. There are certain things, such as ACID
transactions, which are *impossible* to geographically distribute with
redundancy in a performant and scalable way because of speed of light
constraints.

Of course web-startups like Reddit have no excuse in this area: they
don't even *need* ACID transactions for anything they do, as what they
are storing is utterly unimportant in the financial sense and can be
handled with eventually-consistent semantics. But asynchronous
replication doesn't cut it for something like stock trades, or even
B2C order taking.

I like to bag on my developers for not knowing anything about the
infrastructure, but sometimes you just can't do it right because of
physics. Or you can't do it right without writing your own OS,
networking stacks, file systems, etc., which means it is essentially
"impossible" in the real world.