Low Cost 10G Router

Tue May 19 18:46:36 UTC 2015

How cheap is cheap and what performance numbers are you looking for?

About as cheap as you can get:

For about $3,000 you can build a Supermicro OEM system with an 8-core Xeon
E5 V3 and 4-port 10G Intel SFP+ NIC with 8G of RAM running VyOS.  The pro
is that BGP convergence time will be good (better than a 7200 VXR), and
number of tables likely won't be a concern since RAM is cheap.  The con is
that you're not doing things in hardware, so you'll have higher latency,
and your PPS will be lower.

I haven't tried this configuration as a full router in production, but have
been using them in a few places as a firewall solution and they've handled
everything I've thrown their way so far.  Initially, I had these in place
as "low-capital" solutions that were going to be temporary so we could
start building out a new environment and collect usage data to have real
world sizing data for something like an ASA cluster, but they've worked so
well that we've held off on that purchase for now (given challenging budget
times in higher-education).

The stability of VyOS has been good, and the image-based upgrade system has
worked every time without issues for the past year or two (starting from
1.0.1 to the current 1.1.5).  That said documentation for VyOS is poor, so
you should be ready to dig into some source code or hit the IRC channel to
get things running.  Having a foundation with general Linux knowledge is
helpful here too.

If you just need a 10G link but only commit to 2-3G then this solution
might be able to work well for you.  If you need closer to line-rate 10G at
small packet sizes then you might start running into performance
limitations due to latency.  If this is the case there is the Vyatta
vRouter 5600 (VyOS is based on the GPL portions of the 5400), which claims
to have Intel DPDK support and can handle multi-10G at line rate; but last
time I checked it was really expensive ($10,000 per core or something
ridiculous like that).

In terms of commercial solutions, I think 10G and BGP are two things that
don't combine well for "cheap".

An ASR1K might do the trick, but more likely than not you're looking at an
ASR9K if you want full tables; I don't have any experience with the 1K
personally so I can't speak to that.  The ASR 9K is a really great platform
and is what we use for BGP here, but it's pretty much the opposite of cheap.

As far as the firewall stuff goes, I have a draft of VyOS as a firewall
that I've been wanting to put together (still needs work):

http://soucy.org/vyos/UsingVyOSasaFirewall.pdf

P.S. Sorry the documentation for VyOS is so bad, what's there so far in the
User Guide is basically me trying to do a first pass in hopes that others
would help out and there haven't been many updates.

On Tue, May 19, 2015 at 1:22 PM, Colton Conor <colton.conor at gmail.com>
wrote:

> What options are available for a small, low cost router that has at least
> four 10G ports, and can handle full BGP routes? All that I know of are the
> Juniper MX80, and the Brocade CER line. What does Cisco and others have
> that compete with these two? Any other vendors besides Juniper, Brocade,
> and Cisco to look at?
>

-- 
Ray Patrick Soucy
Network Engineer
University of Maine System

T: 207-561-3526
F: 207-561-3531

MaineREN, Maine's Research and Education Network
www.maineren.net