BDP discussion pointers: was: Re pontification bloat (was 10GE TOR port buffers (was Re: 10G switch recommendaton))

Jim Gettys jg at
Sat Jan 28 15:45:03 UTC 2012

On 01/28/2012 10:28 AM, Jim Gettys wrote:
> On 01/27/2012 08:31 PM, Randy Bush wrote:
>>>> for those who say bufferbloat is a problem, do you have wred enabled
>>>> on backbone or customer links?
>>> For *most backbone networks* it is a no-op on the backbone.  To be
>>> more precise, if the backbone is at least 10x, and preferably more
>>> like 50x faster than the largest single TCP flow from any customer
>>> it will be nearly impossible to measure the performance difference
>>> between a short FIFO queue and a WRED queue.
>> when a line card is designed to buffer the b*d of a trans-pac 40g, the
>> oddities on an intra-pop link have been observed to spike to multiple
>> seconds.
> See the  CACM article Bufferbloat: Dark Buffers in the Internet, in the
> January CACM, by Kathy Nichols and myself or online at:
> The section entitled "Revisiting the Bandwidth Delay Product" is germain
> to the discussion here. Fundamentally, the b*d "rule" isn't really very
> useful under most circumstances, though it helps to understand what it
> tells you, and may be a useful upper bound under some circumstances,
> though very seldom for a network operator such as found on the NANOG list.
> The fundamental problem is most people don't know either the bandwidth,
> nor the delay. 
> The BDP is what you need for a single long lived TCP flow; as soon as
> you have multiple flows, it's over-estimating the buffering needed, even
> if you know the bandwidth and the delay...
> And this work in particular for routers (or potentially switches) is
> important:
> Appenzeller, G., Keslassy, I., McKeown, N. 2004. Sizing router buffers.
> ACM SIGCOMM, Portland, OR, (August).
> <>
> Ultimately, we need an AQM algorithm that works so well,  and requires
> no configuration, so that we can just always have it on and we can
> forget about it; (W)RED and friends aren't it; but it's the best we've
> got for the moment that you can actually use.  It's hopeless to try to
> use it in the home, where we have very highly variable bandwidth.
> In backbone networks, the biggest reason I can see for enabling (W)RED
> may be for robustness sake: if you have a link and it congests, you can
> quickly be in a world of hurt.  I wonder what happened in Japan after
> the earthquake....  And it should always be on on congested links you
> know about, of course.

Also in particular see section 3.1 in the Sizing Router Buffers paper:
if you don't have AQM enabled, *and* your router/switch is a bottleneck
link, multiple long lived TCP flows will synchronise and you again need
a full BDP sized buffer; this is why random drop in AQM algorithms is

So your millage will vary.  BDP really isn't useful most of the time,
other than thinking about the problem in the first place.
                        - Jim

More information about the NANOG mailing list