Part of the state of Hawaii off-line

Patrick J. Chicas pjc at off-road.com
Tue Sep 23 22:39:24 UTC 1997


On Tue, 23 Sep 1997, Sean Donelan wrote:

> For network purposes, I guess Hawaii counts as part of North America.

For quite some time.. :-)

> 
> Yet Another Utility Explosion.
> 
> A utility explosion has effected electrical service to the building
> housing several of the inter-state providers for Hawaii.  Is anyone
> keeping statistics on these?

A little more info..

The outage started at 1400hrs HST, 9/22.

The power vault fire cause is suspect to be rats and/or termites that ate 
through insulation.

The outage darkened a particular building that houses the following carriers:

-MCI International
-MCI Telecomm
-GST Telecomm (CLEC)
-Hawaii On-line (largest ISP in State and subsidiary of GST)
-Lava.net (large Oahu ISP)

Of the telco carriers, only MCI-Telecomm was unprepared. A portable 
generator was installed and activated with fire escape wire runs at or 
around midnight. MCI-Telecomm voice trunks were maintained by a battery 
string 'till the generator arrived. All parties co-located at MCI-Telecomm, 
including the ANS-CORE, core router appeared to be completely 
unprotected and failed immediately.

Our Web server is connected via ANS. Our Primary DNS and mail list 
machine via Hawaii On-Line. Hawaii Online restored at 0800hrs on 
9/23. Needless to say, we were off the air.

When working with ANS during restoration, the tech mentioned that 
MCI-Telecomm was to provide generator backup at all ANS sites per contract.
MCI-Telecomm has never, ever had such facilities at that location.

The other major NSP in the islands is UUNET via a contract agreement with 
GTE Internet Services. That POP is located in a true, telecomm bunker 
with fully redundant, stand alone power systems, and diverse carrier - 
diverse entrance route fiber facilities.

The power vault fire was the third such outage in the past 12-18 months.
In the last event, a manhole cover was propelled 30 feet skyward when 
contained gasses ignited in a vault. The utility quickly replaced all 
covers with vented grates to prevent such explosions.  Do you think they 
anticipated more vault fires.....

One other building owner in town has already taken these events to mind 
and has installed diverse power feeds from the local utility to 
compliment a huge stand alone generator plant. The same building will soon 
have diverse route fiber feeds from multiple carriers. Interesting 
enough, the high voltage feed from the generator to the building buss, is 
via metal clad conductors. 

We have yet to see termites eat metal here on the rock..

PJC



More information about the NANOG mailing list