Conflicts and fiber cuts
n5d9xq3ti233xiyif2vp at protonmail.ch
Mon Mar 7 10:00:17 UTC 2022
On Sunday, March 6th, 2022 at 23:40, Sean Donelan <sean at donelan.com> wrote:
> Historically, the largest telecommunication outages have been due > to operator error
Yeah, tell me about it.
In the very recent past there was a certain Tier 1 operator who decided to move a core router between X & Y within the same datacentre. As you might imagine, this impacted a good few hundred fibres.
Physical move happened without issue, router powered up fine, but a large subset of fibres refused to come up.
Cue escalation, lots of testing, lots of talking to vendors etc.
Eventually 72 hours later the problem was solved.
Cause ? SFP signals too high at new location, attenuators fitted, everything came up.
More information about the NANOG