OUTAGE: MCI/Worldcom frame-relay network

Mon Aug 9 07:33:31 UTC 1999

  http://cbs.marketwatch.com/archive/19990806/news/current/wcom.htx?source=blq/yhoo&dist=yhoo

  It would seem that WCOM has indeed been communicating with the
  public, as early as 13:33 EST, August 6th.

  This is my concern with the recent trend in using NANOG
  as a pulpit for bringing focus to certain network problems.

  The information displayed here is not always accurate, and
  often subjective.

  I do not mean to suggest that Sean is, only that it does 
  happen, and the list goes for a while with no correction.

  Usually I'd argue that any information is better than no
  information, but when the quality is of suspect value, I'm
  not sure the viewpoint stands.

  I'd recommend that an alternate forum be used for outage
  notification, like Stan Barber's outage-discuss list.

  -alan

Thus spake Sean Donelan (SEAN at SDG.DRA.COM)
 on or about Mon, Aug 09, 1999 at 01:12:47AM -0500:
> 
> SEAN at dra.COM (Sean Donelan) writes:
> >I'm trying to post no more than once in 12 hours.
> 
> Its been another twelve hours.
> 
> Although I don't like all the red on our network map, it is showing how
> resiliant IP protocols are.  My core network uses several different facility
> providers and technologies, and for those customers who have more than one
> connection: IP works.  How else could I send this e-mail?  However, for
> customers with only one connection, they don't really care how many other
> connections are working.  They only care about their one connection.
> 
> MCI/Worldcom claims 75% up their frame-relay network is 'up.'   This is
> down from their previous claim on Friday that 90% of the frame-relay
> network was 'up.'  We still see a number of PVCs throughout the US and
> Canada either completely down, or dropping so many frames as to be
> unusable.  I received similar reports from other MCI/Worldcom frame-relay
> users throughout the US and even as far away as Germany.
> 
> To try to figure out what is going on MCI/Worldcom imposed a 'quiet time'
> on their network, telling their engineers to stop touching things.  This
> helped somewhat.  Portions of the network stabilized on its own, but there
> is still massive congestion.  The quiet time is over.  Now MCI/Worldcom is
> trying to reduce the congestion by disconnecting parts of the frame-relay
> network.
> 
> Remember this is Sunday, normally a low-traffic day.  Monday is underway
> in Europe, and will be hitting the East Coast of the USA soon.  Traffic
> levels are going to increase on the network, assuming any traffic can
> get through at all.
> 
> MCI/Worldcom's web site continues its long silence about this outage.
> I've got to hand it to MCI/Worldcom; its engineers may have trouble
> keeping a frame-relay network going, but its PR department has done
> a great job keeping a lid on the story.  Even MCI/Worldcom customers
> can't find out what is happening.
> -- 
> Sean Donelan, Data Research Associates, Inc, St. Louis, MO
>   Affiliation given for identification not representation