BBN outage

Sat Oct 12 01:50:45 UTC 1996

On Fri, 11 Oct 1996, Rob Liebschutz wrote:

> I just spoke to their NOC and was told that a power switch that is
> supposed to be able to switch them between 3 different power utilities
> failed at 12:30 this morning (friday).  They bought and installed 2
> large diesel generators today and are hoping to be back up using the
> generators within 30 minutes.

Perhaps there is a lesson in redundancy here. Based on this account it
would appear that although they had arranged for 3 power sources that
there were two mistakes made. One was that switching between sources
relied on a single piece of hardware thus negating the redundancy of 3
sources. The other was that they did not have alternate types of power
source, i.e. they did not have generators on site but expected that 3
different utility power sources was adequate redundancy.

Some NOC engineers (or perhaps NOC managers) aren't paranoid enough.

Michael Dillon                   -               ISP & Internet Consulting
Memra Software Inc.              -                  Fax: +1-604-546-3049
http://www.memra.com             -               E-mail: michael at memra.com