cooling door

Wed Apr 2 10:02:36 UTC 2008

> Eg, I 
> click on a web page to log in. The login process then kicks 
> off a few authentication sessions with servers located 
> halfway around the world. Then you do the data gathering, 2 
> phase locks, distributed file systems with the masters and 
> lock servers all over the place. Your hellish user 
> experience, let me SHOW YOU IT.

I know that this kind of hellish user experience exists today
but in those situations, the users are not happy with it, and
I don't think anybody would consider it to be a well-architected
application. But let's use two concrete examples. One is a city
newspaper hosted in a local ISP data center which interacts with
a database at the newspaper's premises. The other is an Internet
retailer hosted in the same local ISP data center which basically
runs standalone except for sending shipping orders to a fulfillment
center in another city, and the nightly site updates uploaded by
the store owner after returning home from the dayjob.

Now, move the retailer half a mile closer to the city center in
distributed pod 26 of a distributed ISP data center, and move
the newspaper to pod 11 which just happens to be on their own
premises because they have outsourced their data center to the
ISP who runs these distributed data center pods. For the life of
me, I can't see any technical issue with this. Obviously, it 
doesn't suit the customers who can't fit into a single pod because
they risk creating a scenario like you describe. I'm not
foolish enough to suggest that one size fits all. All I am suggesting
is that there is room for a new type of data center that mitigates
the power and cooling issues by spreading out into multiple pod
locations instead of clustering everything into one big blob.

> Other than that minor handwaving, we are all good. Turns out 
> that desining such distributed datacenters and setting up 
> applications that you just handwaved away is a bit harder 
> than it looks. I eagerly await papers on distributed database 
> transactions with cost estimates for a distributed datacenter 
> model vs. a traditional model.

For the big applications which cannot fit inside a single pod,
I expect the Amazon EC2/S3 model to influence the way in which
they approach decentralized scaling. And in that case, when
these people figure out the details, then the distributed pod
data center architecture can support this model just as easily
as the big blob architecture. I'm not holding my breath waiting
for papers because in the real world, people are going with what
works. The scientific world has bought into the grid architecture, 
or the so-called supercomputer cluster model. Everyone else is 
looking to Google MapReduce/BigTable, Amazon EC2/S3, Yahoo Hadoop,
XEN virtualization and related technologies.

> See above. That right there doesn't quite solve most of the 
> problems of traditional jobsets but its kind of hard to hear 
> with the wind in my ears.

This is NANOG. Traditional jobsets here are web servers, blogs/wikis,
Internet stores, Content-management sites like CNN, on-demand video,
etc. The kind of things that our customers run out of the racks that
they rent. Tell me again how on-demand video works better in one big
data center?

> I guess I can try to make it clearer by example: look at the 
> cross-sectional bandwidth availability of a datacenter, now 
> compare and contrast what it would take to pull it apart by a 
> few tens of miles and conduct the cost comparison.

It would be pretty dumb to try and pull apart a big blob architecture 
and convert it into a distributed pod architecture. But it would be
very clever to build some distributed pods and put new customers in
there.

--Michael Dillon