oof. panix sidelined by incompetence... again.

Sun Jan 22 18:09:08 UTC 2006

This is hardly as serious as the last incident -- but, well, some people
do seem to have all the luck, eh?

Of course, there are measures one can take against this sort of thing; but
it's hard to deploy some of them effectively when the party stealing your
routes was in fact once authorized to offer them, and its own peers may
be explicitly allowing them in filter lists (which, I think, is the case
here).  Sometimes "budget" network connectivity isn't -- even when you've
already realized that and turned off the tap!

The text below is what's currently in the MOTD on Panix's NetBSD hosts:

======

Con Ed 'stealing' Panix routes (alexis) Sun Jan 22 12:38:16 2006

   All Panix services are currently unreachable from large portions of the
   Internet (though not all of it). This is because Con Ed Communications,
   a competence-challenged ISP in New York, is announcing our routes to the
   Internet. In English, that means that they are claiming that all our
   traffic should be passing through them, when of course it should not.
   Those portions of the net that are "closer" (in network topology terms)
   to Con Ed will send them our traffic, which makes us unreachable.

   We are taking several steps to deal with this:
   1) We are announcing "more specific" routes to our peers. More specific
   routes are always preferred. However, we have to contact network admins
   at those peers to get them to change their route filters, before this
   workaround will be effective.
   2) We are attempting to reach Con Ed Communications. Unfortunately, so
   far we've been unable to do so. They don't seem to answer their phones
   on Sunday.
   3) We are attempting to reach Verio, which is "upstream" from Con Ed,
   because they could (and should!!) choose to ignore the rogue routes from
   Con Ed.

   Since all of these depend on humans outside of Panix, we can't give a
   specific time at which we expect this problem to be worked around (I
   don't expect a real resolution for a while, because Con Ed is hopeless,
   but the workaround will be perfect until then). But we do expect to
   be able to reach responsible parties at our peers within a few hours at
   most. We don't know how long it will take for them to change their
   filters, but that's not a challenging job technically, so we hope it won't
   take long.

   I'll post another MOTD as soon as we know anything more.