Level 3's side of the story
JC Dill
lists05 at equinephotoart.com
Sat Oct 8 14:24:06 UTC 2005
Eric Louie wrote:
>
> DISCLAIMER: From one of the clueless
As a disclaimer, I will point out that there are some in this debate who
consider me clueless as well. However, I don't believe that any of the
following is in error.
> During this entire debaucle, I never saw any mention of:
I've mentioned most of it in prior posts but perhaps it hasn't been
explained well enough for you to fully grasp it. I hope this helps.
> 1) Cogent sending "transit" traffic to Level3, which leads me to
Cogent was sending peering traffic, not transit traffic.
> believe that all the traffic from Cogent through the peering points was
> actually *destined* for Level3 customers.
Yes, that's what peering is. Peering gets you the other ISP's customers
but NOT all the rest of the internet that is reachable thru that ISP.
Transit gets you all the internet that is reachable thru that ISP.
> Does the routing support this idea?
Yes.
> Is it safe to assume the opposite, also... that only traffic
> destined for Cogent customers came through the Level3 peering points?
Yes.
> And that Level3 had one and only one path to Cogent (no one else
> providing transit for them to Cogent AS'es?)
Yes.
> 2) Level3 making any contingency for their own customers to reach
> Cogent networks (any announcements to their own customers)
L3 made no contingency plans for their customers to reach Cogent (and
visa versa).
For the L3 customers that were multi-homed, they had other paths to
Cogent thru their other service providers. Those that were not
multi-homed had no path to Cogent when L3 cut the peering, because as a
"tier 1" (the widely accepted definition of "tier-1" is that the network
only has SFI (Settlement Free Interchange i.e. peering) links to other
networks and thus no transit links) none of the other networks L3
connects to will carry L3's traffic to a third network (another peer).
Cogent was a "tier 1" until prior de-peering incidents left them unable
to reach other networks. They solved this by buying filtered transit
thru Verio to reach the networks they couldn't reach via peering.
L3 was hoping to force Cogent to increase that transit to include the
traffic destined for L3's customers, thus raising Cogent's transport
costs at no additional (transport) cost to L3.
> 3) Possible traffic issues. Was Cogent guilty of not transporting the
> Level3-bound packets within the Cogent network to the closest
> point-of-entry peer to the host in the Level3 network, therefore
> "costing" Level3 transit of their own packets?
Possible, in fact probable. Most ISPs hand off traffic to peers under a
"hot potato" policy, they hand it off at the closest point where they
connect. If the traffic is equal in both directions then this works.
If the traffic is not equal, then this lowers the cost of the network
that has high outbound traffic, as the other network bears the brunt of
the total cost for transporting the combined traffic between their
respective customers.
> In other words is it also a traffic engineering issue?
Or rather, that it could be mostly solved with a traffic engineering fix
called "cold potato" routing, where the sending network carries the
traffic as far as they can before handing it off to the recipient network.
Consider a simple hypothetical closest-exited network setup (hot potato
routing) between 2 peers:
ISP Eyeballs: Router-E1----2,000 Mile Link----Router-E2----Customer
| |
| |
peering peering
| |
| |
ISP Content: Server--Router-C1---2,000 Mile Link-----Router-C2
When the customer on ISP E (Eyeballs) requests content (web page, music
file, etc.) from the server on ISP C (Content), packets travel like this:
Customer->Router-E2->Router-C2->Router-C1->Server
When the server returns traffic to the customer, traffic goes like this:
Server->Router-C1->Router-E1->Router-E2->Customer
The problem is the customer->server direction would typically be a 500
byte request and 64 byte ACK packets, where as the server->customer data
includes many 1500 byte data packets. So, ISP Eyeballs may carry 2Mbs
of data over its 2,000 mile link, where as ISP Content will only carry
128Kbs over its 2,000 mile link.
Even though both companies met in the middle, ISP Content shifted some
of its costs to ISP Eyeballs.
Back when most ISPs had the same types of traffic (even mixes of content
and eyeballs), they had even ratios which equalized this effect, as it
was happening the same amount in both directions. But as some ISPs
started specializing in one type of content or the other, uneven flows
were produced. Some bean-counter felt that these uneven flows meant
that the network that was sending more traffic should now pay for
transit, even though this traffic was traffic that their own customers
were requesting and paying them to transmit!
There are ways to deal with it though, like cold potato routing.
> Are some of the business issues solvable by proper engineering and
> filtering (or statistics-jockeying)?
Yes, see above.
jc (no coffee yet, errors possible)
More information about the NANOG
mailing list