Software router state of the art
sharp at sharpone.net
Fri Jul 25 15:44:15 UTC 2008
Yes. We put in some Vyatta routers to extend our corporate network into
another building as a temporary solution (the building had a very short
lease, so our boss didn't want to spend any money on Juniper which is
our usual net gear vendor). Consequently, we are still there.. go figure.
When we started w/ them, they were still using the XORP routing engine
(and we haven't upgraded to the new platform yet). My experience wasn't
terribly good. The first issue was a bad memory leak in the router
manager process when VRRP hello times were set to 1 second. The first
indication of something wrong is that our master router crashed,
followed by his backup. Had to physically reboot the boxes to get them
back online, which involved driving there as no one onsite had access to
the cage at the office. All voice and data ran through these routers,
basically rendering every employee useless until we got it back online.
It wasn't a happy day. After that we had to monitor memory and do
controlled reboots every month or so. We eventually convinced Vyatta of
this memory leak and they were able to fix it, but that was a very
frustrating process, and time consuming for us, which is why the next
problems I describe, we have just found our own workarounds.
The next problem was a combination of a problem with the Vyatta and a
problem w/ our IP phones. The Vyatta was sending garp's for the data
vrrp address out the voice vlan (same 2 routers are default gate on both
data and voice vlans). All of the workstations run through the phones
(which sit tagged on voice vlan, and pass traffic from workstation
untagged to data vlan). The phone, seeing the arp for the data vrrp
address on its voice vlan, would send traffic to that address out the
voice vlan, effectively taking that workstation off the net for anything
other than local traffic. That was a bugger to figure out, and basically
we solved it with an arptables rule on the vyatta's. That was the one
advantage of using a Linux (debian) based router platform, was that we
could load other 'unsupported' packages to solve problems like this.
The last thing is that OSPF never really converges correctly. You can
view the OSPF database, and see which default the routers should
converge to, but they do not. They will sit converged to one path for a
while, and if you reboot the other router that generates default, they
will reconverge to it for a while. This hasn't been a big enough problem
for me to worry about it.
Last thing to say is, I haven't tried upgrading since Vyatta abandoned
the XORP platform and moved to the Quagga platform, but I'm guessing
(based on experience w/ Quagga) that they have a lot fewer of these
quirks that I've described.
IMHO, YMMV, etc
Tim Sanderson wrote:
> Is anyone using Vyatta for routing? I sure would like to know about any experience with it in production.
> Tim Sanderson, network administrator
> tims at donet.com
> -----Original Message-----
> From: randal k [mailto:nanog at data102.com]
> Sent: Wednesday, July 23, 2008 1:46 PM
> To: Adrian Chadd
> Cc: nanog at merit.edu
> Subject: Re: Software router state of the art
> That is a very interesting paper. Seriously, 7mpps with an
> off-the-shelf Dell 2950? Even if it were -half- that throughput, for a
> pure ethernet forwarding solution that is incredible. Shoot, buy a
> handful of them as hot spares and still save a bundle.
> Highly recommended reading, even if (like me) you're anti-commodity routing.
> On Wed, Jul 23, 2008 at 10:17 AM, Adrian Chadd <adrian at creative.net.au> wrote:
>> On Wed, Jul 23, 2008, Charles Wyble wrote:
>>> This might be of interest:
>> Various FreeBSD related guys are working on parallelising the forwarding
>> layer enough to use the multiple tx/rx queues in some chipsets such as the
>> Intel gig/10ge stuff.
>> 1 mil pps has been broken that way, but it uses lots of cores to get there.
>> (8, I think?)
>> Linux apparently is/has headed down this path.
More information about the NANOG