10GE TOR port buffers (was Re: 10G switch recommendaton)

Sat Jan 28 12:38:21 UTC 2012

On (2012-01-28 21:06 +0900), Masataka Ohta wrote:

> The required amount of memory is merely 150KB.

Assuming we don't support jumbo frames and switch cannot queue sub packet
sizes (normally they can't but VXR at least has 512B cell concept, so
tx-ring is packet size agnostic, but this is just PA-A3)

> If you have 10 classes, it is still 1.5MB.

Yup, that's not bad at all in 100M port, infact 10 classes would be quite
much.

> > And if switch does support QoS but operator configures only BE, and
> > operator does not limit BE queue size, operator will see buffer bloat,
> 
> 1.5MB @ 10Gbps is only 1.2ms, which is not buffer bloat.

You can't buffer these in ingress or you risk HOLB issue, you must buffer
these in the egress 100M and drop in ingress if egress buffer is full.

But I fully agree, it's not buffer bloat. But having switch which does
support very different traffic rates in ingress and egress (ingress could
even be LACP, which further mandates larger buffers on egress) and if you
also need to support QoS towards customer, the amount of buffer quickly
reaches the level some of these vendors are supporting.
When it becomes buffer bloat, is when inexperienced operator allows all of
the buffer to be used for single class in matching ingress/egress rates.

-- 
  ++ytti