<div dir="ltr">Now if you call into CL you get a message stating their technicians are working on an ip outage. </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Aug 30, 2020 at 6:56 AM Chase Christian <<a href="mailto:madsushi@gmail.com">madsushi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">Multiple BGP sessions with Level3 (DIA) started flapping at approx 03:00 Pacific:<div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font face="monospace">Aug 30 03:05:13 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor 4.35.X.Y (AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes</font></div><div><font face="monospace">Aug 30 03:05:13 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event HoldTime new state Idle</font></div><div><font face="monospace">Aug 30 03:07:37 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state OpenConfirm event RecvKeepAlive new state Established</font></div><div><font face="monospace">Aug 30 03:15:38 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event HoldTime new state Idle</font></div><div><font face="monospace">Aug 30 03:17:15 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state OpenConfirm event RecvKeepAlive new state Established</font></div><div><font face="monospace">Aug 30 03:19:55 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor 4.35.X.Y+52091 (proto) 6/7 (Cease/connection collision resolution) 0 bytes</font></div><div><font face="monospace">Aug 30 03:20:11 rtr02 Rib: %BGP-3-NOTIFICATION: received from neighbor 4.35.X.Y (AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes</font></div><div><font face="monospace">Aug 30 03:20:11 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event RecvNotify new state Idle</font></div><div><font face="monospace"><br></font></div></blockquote>And incoming traffic from AS3356 and AS209 both dropped to very low volumes.<font face="monospace"><br></font></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Aug 30, 2020 at 5:58 AM Jason Kuehl <<a href="mailto:jason.w.kuehl@gmail.com" target="_blank">jason.w.kuehl@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">Well, When I tried calling I got a fast busy, so that's nice.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Aug 30, 2020 at 8:33 AM David Hubbard <<a href="mailto:dhubbard@dino.hostasaurus.com" target="_blank">dhubbard@dino.hostasaurus.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div>
<p class="MsoNormal">Same. Also, as reported on outages list, what’s even worse is that they appear to be continuing to propagate advertisements from circuits whose sessions have been turned down. I validated ours still were via a couple looking glass portals.
Down Detector shows nearly every major service provider impacted.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">They’re not reachable so who knows if they’re even working on it. I feel like they’ve been cutting heavily on the network ops side in recent years…<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">NANOG <nanog-bounces+dhubbard=<a href="mailto:dino.hostasaurus.com@nanog.org" target="_blank">dino.hostasaurus.com@nanog.org</a>> on behalf of Drew Weaver via NANOG <<a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a>><br>
<b>Reply-To: </b>Drew Weaver <<a href="mailto:drew.weaver@thenap.com" target="_blank">drew.weaver@thenap.com</a>><br>
<b>Date: </b>Sunday, August 30, 2020 at 8:23 AM<br>
<b>To: </b>"<a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a>" <<a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a>><br>
<b>Subject: </b>Centurylink having a bad morning?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Hello,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Woke up this morning to a bunch of reports of issues with connectivity had to shut down some Level3/CTL connections to get it to return to normal.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">As of right now their support portal won’t load: <a href="https://www.centurylink.com/business/login/" target="_blank">
https://www.centurylink.com/business/login/</a><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Just wondering what others are seeing.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr">Sincerely,<br> <br>Jason W Kuehl<br>Cell 920-419-8983<br><a href="mailto:jason.w.kuehl@gmail.com" target="_blank">jason.w.kuehl@gmail.com</a></div>
</blockquote></div>
</blockquote></div>