Software router state of the art

Justin Sharp sharp at sharpone.net
Fri Jul 25 15:44:15 UTC 2008


Yes. We put in some Vyatta routers to extend our corporate network into 
another building as a temporary solution (the building had a very short 
lease, so our boss didn't want to spend any money on Juniper which is 
our usual net gear vendor). Consequently, we are still there.. go figure.

When we started w/ them, they were still using the XORP routing engine 
(and we haven't upgraded to the new platform yet). My experience wasn't 
terribly good. The first issue was a bad memory leak in the router 
manager process when VRRP hello times were set to 1 second. The first 
indication of something wrong is that our master router crashed, 
followed by his backup. Had to physically reboot the boxes to get them 
back online, which involved driving there as no one onsite had access to 
the cage at the office. All voice and data ran through these routers, 
basically rendering every employee useless until we got it back online. 
It wasn't a happy day. After that we had to monitor memory and do 
controlled reboots every month or so. We eventually convinced Vyatta of 
this memory leak and they were able to fix it, but that was a very 
frustrating process, and time consuming for us, which is why the next 
problems I describe, we have just found our own workarounds.

The next problem was a combination of a problem with the Vyatta and a 
problem w/ our IP phones. The Vyatta was sending garp's for the data 
vrrp address out the voice vlan (same 2 routers are default gate on both 
data and voice vlans). All of the workstations run through the phones 
(which sit tagged on voice vlan, and pass traffic from workstation 
untagged to data vlan). The phone, seeing the arp for the data vrrp 
address on its voice vlan, would send traffic to that address out the 
voice vlan, effectively taking that workstation off the net for anything 
other than local traffic. That was a bugger to figure out, and basically 
we solved it with an arptables rule on the vyatta's. That was the one 
advantage of using a Linux (debian) based router platform, was that we 
could load other 'unsupported' packages to solve problems like this.

The last thing is that OSPF never really converges correctly. You can 
view the OSPF database, and see which default the routers should 
converge to, but they do not. They will sit converged to one path for a 
while, and if you reboot the other router that generates default, they 
will reconverge to it for a while. This hasn't been a big enough problem 
for me to worry about it.

Last thing to say is, I haven't tried upgrading since Vyatta abandoned 
the XORP platform and moved to the Quagga platform, but I'm guessing 
(based on experience w/ Quagga) that they have a lot fewer of these 
quirks that I've described.

IMHO, YMMV, etc

--Justin

Tim Sanderson wrote:
> Is anyone using Vyatta for routing? I sure would like to know about any experience with it in production.
>
> http://www.vyatta.com/
>
> --
> Tim Sanderson, network administrator
> tims at donet.com
>
>
> -----Original Message-----
> From: randal k [mailto:nanog at data102.com]
> Sent: Wednesday, July 23, 2008 1:46 PM
> To: Adrian Chadd
> Cc: nanog at merit.edu
> Subject: Re: Software router state of the art
>
> That is a very interesting paper. Seriously, 7mpps with an
> off-the-shelf Dell 2950? Even if it were -half- that throughput, for a
> pure ethernet forwarding solution that is incredible. Shoot, buy a
> handful of them as hot spares and still save a bundle.
>
> Highly recommended reading, even if (like me) you're anti-commodity routing.
>
> Cheers,
> Randal
>
> On Wed, Jul 23, 2008 at 10:17 AM, Adrian Chadd <adrian at creative.net.au> wrote:
>   
>> On Wed, Jul 23, 2008, Charles Wyble wrote:
>>
>>     
>>> This might be of interest:
>>>
>>> http://nrg.cs.ucl.ac.uk/mjh/tmp/vrouter-perf.pdf
>>>       
>> Various FreeBSD related guys are working on parallelising the forwarding
>> layer enough to use the multiple tx/rx queues in some chipsets such as the
>> Intel gig/10ge stuff.
>>
>> 1 mil pps has been broken that way, but it uses lots of cores to get there.
>> (8, I think?)
>>
>> Linux apparently is/has headed down this path.
>>     
>
>
>   





More information about the NANOG mailing list