Router crash unplugs 1m Swedish Internet users

Vadim Antonov avg at kotovnik.com
Tue Jun 24 02:06:37 UTC 2003



On Mon, 23 Jun 2003, Jim Deleskie wrote:

> One router and it takes there entire network off-line... Maybe someone needs
> a Intro to Networks 101 class.

No matter what kind of technology or design you have there are always
kinds of faults which may bring the entire system down.  The problem is
generally in recognizing when a fault has occured, so the the operation
may be switched over to a backup.

Particularly, the present Internet routing architecture is (mis)designed
in such a way that it is incredibly easy for a local fault or human error
to bring a significant portion of the network down.  Even single-box
_hardware_ faults may lead to global crashes.

Long long time ago I had to track down a problem which made US and EU
pretty much disconnected for several hours. This turned out to be a
hardware problem in 7000's SSE card, which happily worked with packets
originating and terminating in the router itself, but silently dropped all
transit packets.  Voila!  Neighbour boxes were convinced that this one's
working - because all routing protocols were happy, and were trying to
send lots of traffic through it, which was simply going to a blackhole to
the mighty annoyance of everyone.  I've got a speeding ticket showing over
100mph on Dulles hwy at 3am, too, as a memento of rushing to DC with a
spare card...

So, in the absense of details, I would reserve judgement on soundness of
design practices.

--vadim




More information about the NANOG mailing list