Charter regional(nationwide?) flapping/multi outages

Tue Apr 3 08:27:56 UTC 2012

[ This email takes place and context between 0817 GMT and 0910 GMT ]

Charter is/was/has been/may still be hit by regional to national outages,
starting ~ 0817 GMT

Not only is my home ofc (100mb, quad doc3/rg6, hangs off chi) down (dying
well within the network and not at cpe-adjacent gear), Charter NOC and
Eng's cant even get to their ticketing and status/testing systems. They're
dead in the water. (Voice service aside)

... :

Three thoughts come to mind.

1) Tech says Charter (according to internal talk) has no v6 deploy plans
until 2013.  Someone stop me from pulling out my hair on this -- Does 3q
'13 align with others' plans for v6 deployment ?

2) Eating your own dogfood is awesome, but where is a backup plan?  My
traces out during the ~30 mins on the horn had me routing thru Chi, Cle,
and MO, dying at border/cores every time. Tethering my laptop to my
android, I saw similarly-stopping routes inbound.  (BGPlay disagrees, but
thats another issue).
    Does it not behoove call centers and NOCs to have local access to
replicated ticket and status dbs, failing over to alt carriers during
severe outages (or any outage that takes down primary support)?

3) The first line tech suggested "it's DNS" (yet I run two of my own
nameservers @ home, and roll neustar for global) -- Are we (senior types)
just trying to get nocs off the phone with whatever answer, even if it
involves lies that (we're naive to think) there /aren't/ those without clue
that will challenge this, from premise to organization,
sometimes *(cough)*. bringing these issues to a national stage?

Thoughts, comments, insults, jokes, bring it.  Anonymization assured should
you want to go OTR and have me repost.