Curiosity about AS3356 L3/CenturyLink network resiliency (in general)

Mon May 21 01:04:57 UTC 2018

To circle back to the original post... Level 3 does have multiple routes out of Tampa. They just apparently don't use them all for their transit service. Why not? 

----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

----- Original Message -----

From: "valdis kletnieks" <valdis.kletnieks at vt.edu> 
To: "Baldur Norddahl" <baldur.norddahl at gmail.com> 
Cc: nanog at nanog.org 
Sent: Sunday, May 20, 2018 5:43:42 PM 
Subject: Re: Curiosity about AS3356 L3/CenturyLink network resiliency (in general) 

On Sun, 20 May 2018 09:16:25 +0200, Baldur Norddahl said: 

> He is complaining about AS3356 in specific and claiming they COULD 
> reroute around it but choose not to. This leads me to assume there are 
> alternatives. Two places, Miami and Texas, are mentioned and that a 
> double fault, one in Miami and another in Texas would bring down the 
> network. I am from Europe, but am I to believe that Miami and Texas (or 
> anywhere between those two) are served by only two fiber conduits? 

There's a difference between "route around it by flipping some BGP magic" and 
"route around it by digging a ditch to a third city". 

The fact that other places have other conduits doesn't change the fact that a 
given city may only have two physical conduits handy. Often, there are other 
*possible* paths that could be built out, but other providers have looked at 
the cost of digging a ditch from the city, out a third path, to their closest 
POP, and decided it's not economically feasible. You can only route across the 
fiber that's actually there and lit up. 

You're from Europe? OK, consider this setup: Andorra. Two providers, one of 
who backhaul that path all the way to Madrid, and the other that backhauls to 
Marseilles. Sure, there's other cities along the way, but there's no fiber path 
from where you are to there. For instance, the fiber path may run from Madrid 
to Zaragoza, where it splits 3 ways to Pamplona, Andorra, and Barcelona - but 
if Barcelona and Pamplona don't provide alternate paths out to the net, you're 
still going to Madrid. Meanwhile, other companies may provide service to lots 
of smaller places along the border on the Spain side, and other companies 
provide service to lots of places on the French side, but not into Andorra 
itself. 

You don't like that, consider any one of the many European cities that are in a 
deep river valley, so the only realistic ways to the outside world are 
"upstream" and "downstream". 

> The question was if downtime on a transit provider of many hours is 
> unacceptable. I am offering my experience that this happens to all of 
> them. Some of them can have problems that last days not hours. Do not 
> ever assume that a so called "tier 1" network is good as your only transit. 

The gotcha here is the very high danger than with only two paths out of the 
city, your second and third choices are fate-sharing with that Tier 1. If you're 
in Andorra, and you have 8 providers that share a path through a tunnel to Toulouse, 
and another 6 that share a bridge to Barcelona, you still have a problem. 

(That, and anybody who buys transit only from one Tier 1 is going to have 
a really hard time getting routes to the *rest* of the internet...) 