Fiber cut in SF area
Patrick W. Gilmore
patrick at ianai.net
Fri Apr 10 15:19:25 CDT 2009
On Apr 10, 2009, at 3:41 PM, Scott Doty wrote:
> George William Herbert wrote:
>> Scott Doty wrote:
>>> (Personally, I can think of a "MAE-Clueless" episode that was
>>> worse than this, but that was in the 90's...)
>> The gas main strike out front of the building in Santa Clara?
>> Or something else?
>> -george william herbert
>> gherbert at retro.com
> No, it was when an AS took their full bgp feed & fed it into their
> igp (which used RIP, iirc), which generated (de-aggregated) routes
> into /24's, which they then announced back into bgp...
That was Vinny Bono of FLIX, the Fat man Little man Internet eXchange,
as7007. Happened in 1997, IIRC. He used a Bay Networks router to
redistribute BGP on one card into RIPv1 on another card, stripping the
CIDR notations off each prefix, making them classful, and stripping
the AS Path. This means, for instance, 22.214.171.124 was a /8, not a /24.
It also means He then re-redistributed RIP into BGP on a third card,
which then originated each route from as7007.
I have it on most excellent authority (the "Fat man" himself) that
this was not possible on ciscos. Wonder if it is now ... ?
Anyway, I did not know people were calling this the "MAE-Clueless"
incident. I've always called it the "7007 incident". In fact, some
people still have as7007 filtered.
> iirc, part of the chaos than ensued was due to a router bug, so that
> the routes "stuck around" in global views, even after the AS killed
> their announcements, and even after physically disconnecting from
> their provider.
That was Sprint, as7007's transit provider. Sprint only did AS Path
filtering, and as every single prefix was ^7007$, they all passed the
Vinny literally unplugged the router, no power, no fiber, no copper,
but the prefixes were still bouncing around the 'Net for hours.
Sprint kept the routes around for a long time as their routers would
not honor withdrawals - or so the rumors said. The rumors also
claimed the IOS version was named "$FOO-sean". Sean Doran was CTO of
Sprint's Internet company at the time, and he supposedly specifically
asked for the 'feature' of ignoring withdrawals to lower CPU on their
AGS+s. I have absolutely no way of confirming this as I haven't
spoken to Sean in years & years, and wouldn't even know where to find
him any more.
The most interesting rumor I heard is that Sprint had to shut down
every single router simultaneously to clear the routes out of their
network. Personally I think that's probably a bit exaggerated, but
> We told our customers "the Internet is broken, please try again
> later"...which was acceptable back then. (But I doubt we would get
> away with just that nowadays... ;-) )
Really? That's what some broadband providers say nearly daily.
More information about the NANOG