NAP/ISP Saturation WAS: Re: Exchanges that matter...

Mike Leber mleber at
Mon Dec 16 19:52:14 UTC 1996

On Sun, 15 Dec 1996, Forrest W. Christian wrote:
> By 
> contrast, I can ping any sprint-customer-attached computer and have 
> almost 0% packet loss (1 out of 1000 lost occassionaly).

Providers tend to have better connectivity within their own network.

> IS there a reason that I'm getting 5 - 50% loss outside of sprint?  I've 
> also played a bit with this from a couple of other providers with similar 
> results.

You paint with a pretty wide brush.

The loss is caused by atleast three things:

* ICMP packets are dropped by busy routers

Many routers drop ICMP packets (ping, traceroute) when busy, or alternate
dropping ICMP packets.  I know that this behavior occurs when the packets
are directed to the specific router, I am not sure if this every occurs
for packets passing through.  The standby tool ping needs a more reliable
replacement for testing end to end packet loss.

* Pipe smaller than needed

Some providers have a pipes smaller than they "need" going to the NAPs. 
For a small provider this may be a 10 megabit connection.  For sprint even
a 100 megabit connection may not be enough (they may need multiple
connections).  With the Internet continuing to grow you can expect
periodic growing pains for specific providers that don't forecast far
enough into the future.

* Head of queue blocking in the Gigaswitch

This primarily effects the traffic of specific providers (who have enough
to fill up a 100 megabit pipe).  This phenomenon occurs when your NAP
connection tries to talk to another providers filled up NAP connection. 
Even though the Gigaswitch has input and output queues, your output queue
will block until the other providers input queue is free.  When you block,
you drop packets destined for other potentially available connections.

However, this usually isn't a problem because most providers don't happen
to peer with Sprint (purely an example).  In other words, if you don't
happen to exchange traffic with the overloaded party you won't see the
head of queue problem occur for your packets.

This problem can be fixed by multiple connections to the same NAP.  (Which
many providers already have).

To summarize, the ability of a provider to get packets to and from other
providers is directly dependent on how much money they are willing to
spend to do that.  By necessity they improve their internal network first. 


+------------------- H U R R I C A N E - E L E C T R I C -------------------+
| Mike Leber             Direct Internet Connections     Voice 408 282 1540 |
| Hurricane Electric      Web Hosting & Co-location        Fax 408 971 3340 |
| mleber at                                  |

More information about the NANOG mailing list