<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: arial,helvetica,sans-serif; font-size: 10pt; color: #000000'>It's technical enough so that laypeople immediately lose interest, yet completely useless to anyone that works with this stuff.<br><br><div><span name="x"></span><br><br>-----<br>Mike Hammett<br>Intelligent Computing Solutions<br>http://www.ics-il.com<br><br>Midwest-IX<br>http://www.midwest-ix.com<span name="x"></span><br></div><br><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Saku Ytti" <saku@ytti.fi><br><b>To: </b>"nanog list" <nanog@nanog.org><br><b>Sent: </b>Sunday, December 30, 2018 7:42:49 AM<br><b>Subject: </b>CenturyLink RCA?<br><br>Apologies for the URL, I do not know official source and I do not<br>share the URLs sentiment.<br>https://fuckingcenturylink.com/<br><br>Can someone translate this to IP engineer? What did actually happen?<br>From my own history, I rarely recognise the problem I fixed from<br>reading the public RCA. I hope CenturyLink will do better.<br><br>Best guess so far that I've heard is<br><br>a) CenturyLink runs global L2 DCN/OOB<br>b) there was HW fault which caused L2 loop (perhaps HW dropped BPDU,<br>I've had this failure mode)<br>c) DCN had direct access to control-plane, and L2 congested<br>control-plane resources causing it to deprovision waves<br><br>Now of course this is entirely speculation, but intended to show what<br>type of explanation is acceptable and can be used to fix things.<br>Hopefully CenturyLink does come out with IP-engineering readable<br>explanation, so that we may use it as leverage to support work in our<br>own domains to remove such risks.<br><br>a) do not run L2 DCN/OOB<br>b) do not connect MGMT ETH (it is unprotected access to control-plane,<br>it cannot be protected by CoPP/lo0 filter/LPTS ec)<br>c) do add in your RFP scoring item for proper OOB port (Like Cisco CMP)<br>d) do fail optical network up<br><br>-- <br> ++ytti<br></div><br></div></body></html>