CenturyLink Outage?

Tue May 7 17:17:36 UTC 2013

Outages list is going bananas right now:

Same thing happened in Monroe LA that took down all of north Louisiana. It was an update that went bad and the switch had to be manually checked line by line. No backup was done before the maintenance. 16 hours of downtime about 45 days ago.

On Tuesday, May 7, 2013, Frank Bulk wrote:
Sounds like it was scheduled maintenance gone bad:
        "Perrine said he spoke with CenturyLink at 6 a.m.<x-apple-data-detectors://1>
        where they advised him the company was doing scheduled
        maintenance. At 7:35 a.m.<x-apple-data-detectors://2> they told him something
        had gone wrong during the maintenance and it was
        affecting customers in 13 states."
http://www.tallahassee.com/article/20130507/NEWS/130507010/Centurylink-outag
e-affecting-local-internet-provider?nclick_check=1

I bite my tongue in regards to Network Tallahasee's singlehomedness and
Perrine's comment on his questioning of the timing of the update.

I encourage all my competitors to stay single homed and to do their
maintenance Sunday morning<x-apple-data-detectors://4>, while their vendors are at their lowest staffing
levels.

Frank

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>] On Behalf Of Frank Bulk
Sent: Tuesday, May 07, 2013 8:29 AM
To: 'Marco Prechel'; outages at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>
Subject: Re: [outages] Centurylink nationwide outage

I first saw timeouts with www.qwest.com<http://www.qwest.com/> over IPv6 at 3:08 am Central<x-apple-data-detectors://9> -- I
was wondering why, now I know. =)

Frank

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>] On Behalf Of Marco
Prechel
Sent: Tuesday, May 07, 2013 7:33 AM
To: outages at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>
Subject: [outages] Centurylink nationwide outage

Saw our Centurylink Ethernet circuit lose L3 connectivity in SWFL around
0400 EDT.
ANS/ABS states it's a nationwide routing issue. No ETR.
_______________________________________________
Outages mailing list
Outages at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org<x-apple-msg-load://2CF0901A-5EB4-4331-8B21-8DE35C0C12D0/>
https://puck.nether.net/mailman/listinfo/outages

--
Jeremie Chism
Triton Communications
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

Sent from my iPad

On May 7, 2013, at 10:16 AM, "Jason Lester" <jlester at wcs.k12.va.us<mailto:jlester at wcs.k12.va.us>> wrote:

Does anyone know what is going on with the nationwide CenturyLink outage?
Their NOC recording says it is a BGP routing issue with their upstream
peers affecting Internet traffic and traffic between regions.  Our outside
connectivity with them has basically been down since about 4:00AM (EDT)
this morning.  The prefixes we were receiving from them were fluctuating
between a few hundred and a few thousand all morning.  We are getting the
full BGP table from them now (for about the last hour), but still not
seeing any incoming traffic.  Seems like a major issue since it has been
almost 9 hours now.

Thanks,
Jason
--

Jason Lester
Administrator for Instructional Technology
Washington County Public Schools
Tel: 276-739-3060
Fax: 276-628-1893
http://www.wcs.k12.va.us