Operate until failure

Andy Walden andy at tigerteam.net
Mon Jan 8 15:27:07 UTC 2001


On Mon, 8 Jan 2001, Shawn McMahon wrote:

> We have a generator, but we have had two amazing coincidences cause it
> to fail.  The first time, the generator was fine, but the switch didn't
> switch.  The person who was signing off (erroneously) that he was checking
> that switch monthly lost his job shortly before we stopped using his
> company entirely.  We discovered the problem when the batteries reached
> the point where it was supposed to cut over, and the entire data center
> went dark.  That was a very, very bad day.

I recall that ominous depressing feeling sitting by myself in a dark data
center at 3 in the morning, with nothing but the lights of my equipment,
listening to the rectifiers beeping and as the batteries went dead,
hearing machines drop off one by one until it was completely dark...

andy






More information about the NANOG mailing list