10/17 Fiber Outage

Tue Oct 20 17:56:45 UTC 1992

MCI Fiber Outage - 10/17
================
	At 12:33EST on 10/17 we experienced a major MCI fiber outage
on the east coast that affected the T3 and T1 backbones.  A truck
snagged an overhead fiber-line between Trenton and New Brunswick (New
Jersey).  This caused an extended loss of connectivity for several
east coast circuits transiting the MCI junction in West Orange New
Jersey.  All ckts affected by the fiber cut are back on their original
path as of 10/17/92 22:00 EDT.  The following links were affected:

Link Failure	Outage Time	
============	===============	
C32<>C40	12:33-21:58 EDT
C32<>C56	12:33-21:58 EDT
C32<>C48	15:08-21:58 EDT
C33<>E137	12:33-22:05 EDT	
C48<>C72	12:33-21:04 EDT
C35<>C59        13:59-20:18 EDT
C34<>E164       12:35-20:38 EDT	
nss8<>nss19	12:34-21:00 EDT
nss8<>nss17	12:34-21:00 EDT
nss5<>nss8	12:34-21:00 EDT
nss8<>nss9	12:34-21:00 EDT
nss9<>nss10	12:39-21:01 EDT

Several safetynet links were put into service (on diverse circuit
routes), and backup restoration circuits were provided by MCI during
the fiber outage.  There were some periods of routing instability with
circuits going down and coming back up, and getting congested that
caused temporary loss of connectivity for other network sites as well.
According to our routing logs, the following T3 nodes experienced
outages during this time period.

Oct 17 05:20 E205 - down for 10 minutes
Oct 17 05:56 E205 - down for 15 minutes
Oct 17 04:18 E134,E133 - down for 2 hours and 50 minutes 
Oct 17 07:28 E128 - down for 10 minutes
Oct 17 08:46 E146 - down for 50 minutes
Oct 17 08:10 E136,E145 - down for 1 hour 50 minutes
Oct 17 10:14 E136 - down for 20 minutes
Oct 17 11:10 C56,C57,E146 - down for 20 minutes
Oct 17 19:28 C32,E164 - down for 15 minutes
Oct 17 20:03 C32 - down for 5 hours and 15 minutes
Oct 17 16:34 E137 - down for 8 hours and 50 minutes
Oct 18 01:53 E179 - down for 10 minutes
Oct 18 05:54 E128,E179 - down for 15 minutes

We are conducting a post-mortem with MCI to evaluate the impact of
this outage on the network, and identify any potential areas for
improvement in the future.

	Jordan