massive facebook outage presently
Michael Thomas
mike at mtcc.com
Mon Oct 4 19:55:18 UTC 2021
On 10/4/21 11:48 AM, Luke Guillory wrote:
>
> I believe the original change was 'automatic' (as in configuration
> done via a web interface). However, now that connection to the outside
> world is down, remote access to those tools don't exist anymore, so
> the emergency procedure is to gain physical access to the peering
> routers and do all the configuration locally.
>
Assuming that this is what actually happened, what should fb have done
different (beyond the obvious of not screwing up the immediate issue)?
This seems like it's a single point of failure. Should all of the BGP
speakers have been dual homed or something like that? Or should they not
have been mixing ops and production networks? Sorry if this sounds dumb.
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20211004/182bd366/attachment.html>
More information about the NANOG
mailing list