QOS or more bandwidth

Glen Turner glen.turner at aarnet.edu.au
Thu May 31 08:37:51 UTC 2001

Bill Woodcock wrote:
>     >         Whenever I did the cost of deploying and managing fancy QoS
>     > and compared it with the cost of getting and managing more capacity,
>     > it was always MUCH MUCH cheaper to get and manage more capacity
>     > than to mess with more QoS.
> We did one VoIP network deployment, and I tried each of the different QoS
> services in IOS at that time (about 18 months ago) both in the lab and in
> the field, and more bandwidth was the answer then.

Interesting.  We have a national VoIP network which handles
long-distance calls for the Australian universities.  It's
not a trial, it's a real VoIP rollout that interconnects
the PBXs of the universities.  We think that's about
300,000 handsets.

More bandwidth doesn't cut it, as the voice calls then
fail during DDOS attacks upon the network infrastructure.

It's not too hard to fill even a 2.5Gbps link when new
web servers come with gigabit interfaces and when GbE
campus backbones are being rolled out.  If these DDOS
attacks use protocols like UDP DNS then traffic shaping
the protocol is problematic and the source IP subnet
from where the attack is launched needs to be filtered
instead.  Just finding and filtering a DDOS source
can easily take more than five minutes, which pushes
the availability to less than 99.999% and leads to
legal issues with telecommunications regulators about
access to emergency services.

We found the Cisco low latency queuing to be adequate.
It still has a fair amount of jitter, but not enough to
matter for VoIP calls with a diameter of 4,000Km.

We do have issues with Cisco's 7500: dCEF is still problematic,
but needed for the LLQ feature.  The QoS features are
too linked to the hardware (you can't configure a
service-policy that is enacted on the VIP or main
CPU, depending upon the hardware).  Despite QoS being
most needed on cheap E1/T1 links, they expect you to
upgrade to a many thousand dollar VIP4 to support QoS.

We police access to QoS by source IP address, mapping
non-conforming traffic to DSCP=0.  As this requires an
access list executed for each packet, the number of
VoIP-speaking connecting sites to 50 or so, and requires
H.323/SIP proxies at the edge of the sites.

We don't use IntServ.  When a link is down RSVP
takes so long to find an alternative path that
the telephone user hangs up.

So what we are really waiting for is the implementation
of the combined IntServ/DiffServ model so that hosts on
member networks can do local authentication and bandwidth
reservation and we can police the amount of QoS traffic
presented at each edge interface.

"Local authentication and bandwidth reservation" also
hides a host of issues that are yet to be fully addressed.
Most universities don't even *have* an authentication
source that covers everyone that can use IP telephony.

In practice, even with our deliberately simplistic implementation
this whole area is a IOS version nightmare.  There's no excuse
for a monolithic statically-linked executable in this day and
age.  Hopefully Juniper's success with monolithic kernel and
user-space programs will lead to Cisco looking to leapfrog
the competition and adopt the on-the-fly upgradable software
modules as described in Active Bridging (1997).

Having dissed Cisco, I should point out that their H.323/ISDN
gateway software that runs on their RAS boxes (5300, etc) was
the most solid of all the manufacturers we tested and Cisco was
the vendor most willing to fix the differing interpretations of
ISDN we enountered when we connected the North American-developed
RAS to our European-developed PABXs (Ericsson MD110, Alcatel, etc).


As far as the "engineering staff costs" argument goes, we have found that
they the real engineering time goes into hardening the existing
infrastructure.  When carrying voice every engineering shortcut
you have taken comes back to haunt you -- clocking problems,
a low percentage of CRCs, ATM links with no OAM, synchronous
links with carrier nailed high.

Very little of the costs were due to the configuration of the
routers.  Most of the cost was incurred at the edge of the
network, where there is a greater variety of devices, less
correct engineering, and worse configuration control.

> The trans-Pacific problem is being solved.  Give it another
> 6-18 months and fibre between US/CA and ANZ or JP or SG should drop
> significantly in price.

Although the price will drop, it will never drop to the
levels of trans-Atlantic prices.  Even then, countries
like Fiji which have landing points would find it hard to
raise the millions required to buy an STM-1 from that
landing point.

You can make a case that North America is unique.  It
doesn't have 2,000Km expanses with no population centers.
It isn't split off from the remainder iof the world by
mountain ranges so high that to do an hour's lifting can
take a week.  It doesn't have a population so huge that
training people to lay fiber is in itself a massive

In these cases, no amount of money will buy you more
bandwidth tomorrow and the engineering costs of QoS
and TE are immediately worthwhile.  VoIP and QoS
let you use all of the long-haul optical capacity
for IP, whereas the slower growth of the Internet
in North America allowed the running parallel Internet
and PSTN networks.

 Glen Turner                                 Network Engineer
 (08) 8303 3936      Australian Academic and Research Network
 glen.turner at aarnet.edu.au          http://www.aarnet.edu.au/
 The revolution will not be televised, it will be digitised

More information about the NANOG mailing list