it was damp in belleview
randy at psg.com
Sat Jun 23 18:22:08 UTC 2007
the network at the belleview meeting was fantastic, thanks to the host,
xkl, and the usual suspects (merit, tony, ...).
but there was one outage. as best i know, that outage was caused by one
of the two upstream transit isps bouncing (at least) the nanog prefix on
one of their routers far over the water in seattle.
of course, that is why we had two upstreams. but, flap amplification
and dear old route flap damping caused our prefix to be damped by a
number of global isps. yes, our prefix, not just the path.
for those who want to read up on route flap damping and why this caused
problems, please see <http://www.nanog.org/mtg-0210/flap.html> "Route
Flap Damping: Harmful?" from the 2002 nanog eugene meeting.
for those wishing historical perspective on route flap damping, document
ripe-378 (may 2006) says
In the early 1990s the accelerating growth in the number of
prefixes being announced to the Internet (often due to inadequate
prefix-aggregation), the denser meshing through multiple
inter-provider paths, and increased instabilities started to cause
significant impact on the performance and efficiency of the
Internet backbone routers. Every time a routing prefix became
unreachable because of a single line-flap, the withdrawal was
advertised to the whole core Internet and handled by every single
router that carried the full Internet routing table.
It was soon realized that the increasing routing churn created
significant processing load on routing engines, sometimes
sufficiently high load to cause router crashes.
To overcome this situation RFD was developed in 1993 and has since
been integrated into most router BGP software implementations. RFD
is described in detail in RFC 2439. RFD is now used in many service
provider networks in the Internet.
for reasons described in the 2002 preso cited above, and demonstrated by
the network outage in belleview, ripe-378 goes on to conclude
This Routing Working Group document proposes that with the current
implementations of BGP flap damping, the application of flap
damping in ISP networks is NOT recommended. The recommendations
given in ripe-229 and previous documents  are considered
i.e., it's time to turn it off. you are damaging your customers and
More information about the NANOG