RINA - scott whaps at the nanog hornets nest :-)

Sat Nov 6 20:22:59 UTC 2010

> >
> > Last week I asked the operator of fairly major public peering points
> if they supported anything larger than 1500 MTU.  The answer was "no".
> >
> 
> There's still a metric buttload of SONET interfaces in the core that
> won't go above 4470.
> 
> So, you might conceivably get 4k MTU at some point in the future, but
> it's really, *really* unlikely you'll get to 9k MTU any time in the
> next
> decade.
> 
> Matt

There is no reason why we are still using 1500 byte MTUs at exchange points. 

>From Dykstra's paper (note that this was written in 1999 before wide deployment of GigE):

(quote)

Does GigE have a place in a NAP?

Not if it reduces the available MTU! Network Access Points (NAPs) are at the very "core" of the internet. They are where multiple wide area networks come together. A great deal of internet paths traverse at least one NAP. If NAPs put a limitation on MTU, then all WANs, LANs, and end systems that traverse that NAP are subject to that limitation. There is nothing the end systems could do to lift the performance limit imposed by the NAP's MTU. Because of their critically important place in the internet, NAPs should be doing everything they can to remove performance bottlenecks. They should be among the most permissive nodes in the network as far as the parameter space they make available to network applications.

The economic and bandwidth arguments for GigE NAPs however are compelling. Several NAPs today are based on switched FDDI (100 Mbps, 4 KB MTU) and are running out of steam. An upgrade to OC3 ATM (155 Mbps, 9 KB MTU) is hard to justify since it only provides a 50% increase in bandwidth. And trying to install a switch that could support 50+ ports of OC12 ATM is prohibitively expensive! A 64 port GigE switch however can be had for about $100k and delivers 50% more bandwidth per port at about 1/3 the cost of OC12 ATM. The problem however is 1500 byte frames, but GigE with jumbo frames would permit full FDDI MTU's and only slightly reduce a full Classical IP over ATM MTU (9180 bytes).

A recent example comes from the Pacific Northwest Gigapop in Seattle which is based on a collection of Foundry gigabit ethernet switches. At Supercomputing '99, Microsoft and NCSA demonstrated HDTV over TCP at over 1.2 Gbps from Redmond to Portland. In order to achieve that performance they used 9000 byte packets and thus had to bypass the switches at the NAP! Let's hope that in the future NAPs don't place 1500 byte packet limitations on applications.

(end quote)

Having the exchange point of ethernet connections at >1500 MTU will not in any way adversely impact the traffic on the path.  If the end points are already at 1500, this change is completely transparent to them.  If the end points are capable of >1500 already, then it would allow the flow to increase its packet sizes and reduce the number of packets flowing through the network and give a huge gain in performance, even in the face of packet loss.