BDP discussion pointers: was: Re pontification bloat (was 10GE TOR port buffers (was Re: 10G switch recommendaton))

Sat Jan 28 15:28:08 UTC 2012

On 01/27/2012 08:31 PM, Randy Bush wrote:
>>> for those who say bufferbloat is a problem, do you have wred enabled
>>> on backbone or customer links?
>> For *most backbone networks* it is a no-op on the backbone.  To be
>> more precise, if the backbone is at least 10x, and preferably more
>> like 50x faster than the largest single TCP flow from any customer
>> it will be nearly impossible to measure the performance difference
>> between a short FIFO queue and a WRED queue.
> when a line card is designed to buffer the b*d of a trans-pac 40g, the
> oddities on an intra-pop link have been observed to spike to multiple
> seconds.

See the  CACM article Bufferbloat: Dark Buffers in the Internet, in the
January CACM, by Kathy Nichols and myself or online at:
http://cacm.acm.org/magazines/2012/1/144810-bufferbloat/fulltext
The section entitled "Revisiting the Bandwidth Delay Product" is germain
to the discussion here. Fundamentally, the b*d "rule" isn't really very
useful under most circumstances, though it helps to understand what it
tells you, and may be a useful upper bound under some circumstances,
though very seldom for a network operator such as found on the NANOG list.

The fundamental problem is most people don't know either the bandwidth,
nor the delay. 

The BDP is what you need for a single long lived TCP flow; as soon as
you have multiple flows, it's over-estimating the buffering needed, even
if you know the bandwidth and the delay...

And this work in particular for routers (or potentially switches) is
important:

Appenzeller, G., Keslassy, I., McKeown, N. 2004. Sizing router buffers.
ACM SIGCOMM, Portland, OR, (August).
http://yuba.stanford.edu/~nickm/papers/sigcomm2004-extended.pdf
<http://yuba.stanford.edu/%7Enickm/papers/sigcomm2004-extended.pdf>

Ultimately, we need an AQM algorithm that works so well,  and requires
no configuration, so that we can just always have it on and we can
forget about it; (W)RED and friends aren't it; but it's the best we've
got for the moment that you can actually use.  It's hopeless to try to
use it in the home, where we have very highly variable bandwidth.

In backbone networks, the biggest reason I can see for enabling (W)RED
may be for robustness sake: if you have a link and it congests, you can
quickly be in a world of hurt.  I wonder what happened in Japan after
the earthquake....  And it should always be on on congested links you
know about, of course.

There is hope on this front, but it's early days yet.

                    - Jim