Routed optical networks

Thu May 11 12:15:40 UTC 2023

> On May 11, 2023, at 7:45 AM, Etienne-Victor Depasquale via NANOG <nanog at nanog.org> wrote:
> 
> To clarify the table I linked to in the previous email:
> 
> Cisco estimates IP traffic exchanged over the access network by both businesses and consumers with:
> 
> • endpoints over managed networks and 
> • endpoints over unmanaged networks (“Internet traffic”).
> 
> Both the mobile access network and the fixed access network are considered. 
> 
> Cisco considers IP traffic over managed networks to be characterized by passage through a single service provider. 
> Without explicitly referring to quality of service (QoS), 
> the implication is clearly that the traffic is controlled to meet the QoS demanded by the service level agreement (SLA). 
> 
> In contrast, “Internet traffic” crosses provider domains; 
> typically, this traffic is delivered on the basis of providers’ best effort. 
> These two kinds of traffic complement one another and collectively are referred to as total global IP traffic.

I think there’s a lot of problems here.  While places like my employer will periodically disclose our traffic numbers, and DDoS providers, mitigation platforms and otherwise will disclose the peaks they see, much of this data is a bit opaque, and tools like AI that do in-metro or cross-metro datacenter-datacenter remote DMA type activities, those all count differently.

We have seen a continued trend of the privatization of traffic and localization of that over time.  I’ve watched all the big carriers retreat from their global network reaches to be more of regionalized networks.  A decade ago you would have seen European national incumbents peering and with market in Asia, and the complete global networks continue to shrink.

Meanwhile you have a mix of the content and cloud providers continue to build their business-purpose networks expanding into markets that the uppercase Internet may not need to reach.

You can look at the proposals in the EU about fees, and I have dual thoughts on this which are MY OWN and don’t represent my employer or otherwise, but if you read this post from Petra Arts - https://blog.cloudflare.com/eu-network-usage-fees/ - it speaks around major interconnection points like Frankfurt, which are important but double as problematic.  The number of people that need to go to the near market (eg: Chicago, while I’m in Detroit area) for good connectivity is an issue, meanwhile there’s a robust need to keep traffic within the state of Michigan and a halfway decent ecosystem for that via Detroit IX - (disclaimer, I’m on the board).  There need to be some aggregation points, so not everyone needs to be in Detroit, but also not everyone needs to be in Frankfurt - and content localization needs to continue to happen, but is also very regionalized in popularity.

How to do this all and not have it all route via Chicago or Frankfurt is a challenge, but also not everyone will be in Berlin, Munich or these other markets.  This is where having a robust optical network capability (or backbone) can come into play, that you can deliver deeper from those hub points, but at the same time, I’ve been in meetings where companies have their own challenges accepting that content in those downstream locations as their network was also built to get to/from the major hub cities, or IP space wasn’t allocated in a way that can provide consistent routing results or behaviors.  (This is where IPv6 can be super helpful, it gives the chance to possibly Greenfield, aka not screw it up - at least initially).

There’s huge volumes of IP traffic exchanged, but the largest volumes are being moved over private interconnects or a localized IX to those eyeball networks with the historical global backbones playing more of the long-distance carrier role, which is critical as you want a path to deliver those bits, without it following the ITU-style sender pays model, as the majority of IP traffic is actually requested by the customer of the end-user network.  (All of it if you remove network scans, ddos, web bots/crawlers).

Most networks have no SLA once things cross an unpaid boundary (SFI, or even private peering) - and if they are a customer and that path is congested, it’s up to the customer to upgrade that path.

- Jared (many hats)