CDNs for carriers

Mon Jun 29 13:59:22 UTC 2015

Simple flows wouldn't necessarily tell you if you're pulling a bunch from a Netflix caching box on your upstream somewhere. You'd think you had a huge amount going to your current upstream because technically you do, but a local cache or peer could alter that significantly. As we've been starting up our IX, we're finding that we can send lists of ASNs and prefixes and the various CDNs will tell us how much traffic they see going to our customers. Combine that with what flows tell you and I think you've got a good approach. 

What are some good approaches to determining traffic levels to not only ASNs, but also that ASN's downstream ASNs? You may have ASNs A, B, C, D and E in your flows. Say none of them represent more than 5% of your traffic by themselves. If B, C, D and E all purchase transit from A and you can reasonably peer with A, you actually can move 25% of your traffic over to a peer. Maybe there is no good approach at doing that without a bunch of manual work or paying someone else to do it. 

Looking at some stats from one of our customers that is also going through Equinix Chicago, for their average inbound ~37% of traffic was Netflix, Google was 34% and the next highest was Apple at 5%. Note that Akamai had left Chicago Equinix by this point, so they wouldn't be reflected in those numbers. Those percentages are percent of all traffic they send to Equinix. I believe about 2/3s of their total transit went to Equinix when that got turned up. Their total traffic went up once joining the Equinix IX, presumably because they were now bypassing some congestion somewhere. 

----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest Internet Exchange 
http://www.midwest-ix.com 

----- Original Message -----

From: "Jared Mauch" <jared at puck.nether.net> 
To: "Christopher Morrow" <morrowc.lists at gmail.com> 
Cc: "nanog list" <nanog at nanog.org>, "Ramy Hashish" <ramy.ihashish at gmail.com> 
Sent: Monday, June 29, 2015 8:44:18 AM 
Subject: Re: CDNs for carriers 

> On Jun 29, 2015, at 9:33 AM, Christopher Morrow <morrowc.lists at gmail.com> wrote: 
> 
> On Mon, Jun 29, 2015 at 8:53 AM, Ramy Hashish <ramy.ihashish at gmail.com> wrote: 
>> do you have any figures about how much this 
>> recommended CDN save from the Internet BW? 
> 
> isn't that going to wholey depend on your traffic mix/matrix? 
> Wouldn't it be helpful to look at where your users send/receive 
> traffic and then figure out the best next addition? 
> 
> Maybe your best bet isn't another CDN, but better/more/wider peering 
> with folk 2+ AS hops out from your current next-hop-as set? 

I would say that step 1 is to figure out where your traffic is going. Generically saying “CDN” isn’t enough to know what the results are. 

Once you’ve determined where the traffic is going/coming from you can start to make educated decisions vs just “CDN” guessing. An enterprise profile looks much different than residential for example. 

I recall some companies calling our NOC “under attack” because their software update server went down and the machines failed safe and were all fetching software updates from “the internet” vs the internal caching proxy. 

If you have money to spend, there are a few vendors out there from cheap to $$$$ that will help you look at the traffic to make these decisions. 

If you don’t have money to spend, look at NFSen/pmacct. You may be able to spin up a low-cost VM at your local cloud provider (e.g.: digital ocean). 

Remember to export both your v6 and v4 (ip classic) flows as these can widely differ. 

Look for common ASNs or IP ranges. 

I’m sure there’s numerous consultants on the list that would also assist you in this process. 

Hope this helps. 

- jared 