The Making of a Router

Eduardo Schoedler listas at esds.com.br
Thu Dec 26 19:05:36 UTC 2013


For router with Freebsd+BIRD/Quagga, I suggest BSDRP.http://bsdrp.net



2013/12/26 Ray Soucy <rps at maine.edu>

> You can build using commodity hardware and get pretty good results.
>
> I've had really good luck with Supermicro whitebox hardware, and
> Intel-based network cards.  The "Hot Lava Systems" cards have a nice
> selection for a decent price if you're looking for SFP and SFP+ cards that
> use Intel chipsets.
>
> There might be some benefits in going with something like FreeBSD, but I
> find that Linux has a lot more eyeballs on it making it much easier to
> develop for, troubleshoot, and support.  There are a few options if you
> want to go the Linux route.
>
> Option 1: Roll your own OS.  This takes quite a bit of effort, but if you
> have the tallant to do it you can generally get exactly what you want.
>
> Option 2: Use an established distribution.
>
> Vyatta doesn't seem to be doing much with its FOSS release "Vyatta Core"
> anymore, but the community has forked the GPL parts into "VyOS".  I've been
> watching them pretty closely and helping out where I can; I think the
> project is going to win over a lot of people over the next few years.
>
> http://www.vyatta.org/
> http://www.vyos.net/
>
> The biggest point of failure I've experienced with Linux-based routers on
> whitebox hardware has been HDD failure.  Other than that, the 100+ units
> I've had deployed over the past 3+ years have been pretty much flawless.
>
> Thankfully, they currently run an in-memory OS, so a disk failure only
> affects logging.
>
> If you want to build your own OS, I'll shamelessly plug a side project of
> mine: RAMBOOT
>
> http://ramboot.org/
>
> RAMBOOT makes use of the Ubuntu Core rootfs, and a modified boot process
> (added into initramfs tools, so kernel updates generate the right kernel
> automatically).  Essentially, I use a kernel ramdisk instead of an HDD for
> the root filesystem and "/" is mounted on "/dev/ram1".
>
> The bootflash can be removed while the system is running as it's only
> mounted to save system configuration or update the OS.
>
> I haven't polished it up much, but there is enough there to get going
> pretty quickly.
>
> You'll also want to pay attention to the settings you use for the kernel.
>  Linux is tuned as a desktop or server, not a router, so there are some
> basics you should take care of (like disabling ICMP redirects, increasing
> the ARP table size, etc).
>
> I have some examples in: http://soucy.org/xorp/xorp-1.7-pre/TUNING
> or http://soucy.org/tmp/netfilter.txt (more recent, but includes firewall
> examples).
>
> Also a note of caution.  I would stick with a longterm release of Linux.
>  I've had good experience with 2.6.32, and 3.10.  I'm eager to use some of
> the post-3.10 features, though, so I'm anxious for the next longterm branch
> to be locked in.
>
> If running a proxy server of any kind, you'll want to adjust
> TCP_TIMEWAIT_LEN in the header file and re-compile the kernel, else you'll
> run into ephemeral port exhaustion before you touch the limits of the CPU.
>  I recommend 15 seconds (the default in Linux is 60).
>
> Routing-engine -wise.  I currently have a large XORP 1.6 deployment because
> I have a need for multicast routing (PIM-SM), but XORP is very touchy and
> takes quite a bit of operational experience to avoid problems.  Quagga has
> much more active development and eyeballs.  BIRD is also very interesting.
>  I like the model of BIRD a lot (more of a traditional daemon than trying
> to be a Cisco or Juniper clone).  It doesn't seem to be as far along as
> Quagga though.
>
> One of the biggest advantages is the low cost of hardware allows you to
> maintain spare systems, reducing the time to service restoration in the
> event of failure.  Dependability-wise, I feel that whitebox Linux systems
> are pretty much at Cisco levels these days, especially if running
> in-memory.
>
>
>
>
> On Thu, Dec 26, 2013 at 1:07 PM, jim deleskie <deleskie at gmail.com> wrote:
>
> > I've recently pushed a "large" BSD box to a load of over 300, for more
> then
> > an hour, while under test,  some things slowed a little, but she kept on
> > working!
> >
> > -jim
> >
> >
> > On Thu, Dec 26, 2013 at 1:59 PM, Shawn Wilson <ag4ve.us at gmail.com>
> wrote:
> >
> > > Totally agree that a routing box should be standalone for tons of
> > reasons.
> > > Even separating network routing and call routing.
> > >
> > > It used to be that BSD's network stack was much better than Linux's
> under
> > > load. I'm not sure if this is still the case - I've never been put in
> the
> > > situation where the Linux kernel was at its limits. FWIW
> > >
> > > Jared Mauch <jared at puck.nether.net> wrote:
> > > >Have to agree on the below. I've seen too many devices be so
> integrated
> > > >they do no task well, and can't be rebooted to troubleshoot due to
> > > >everyone using them.
> > > >
> > > >Jared Mauch
> > > >
> > > >> On Dec 26, 2013, at 10:55 AM, Andrew D Kirch <trelane at trelane.net>
> > > >wrote:
> > > >>
> > > >> Don't put all this in one box.
> > >
> > >
> > >
> >
>
>
>
> --
> Ray Patrick Soucy
> Network Engineer
> University of Maine System
>
> T: 207-561-3526
> F: 207-561-3531
>
> MaineREN, Maine's Research and Education Network
> www.maineren.net
>



-- 
Eduardo Schoedler


More information about the NANOG mailing list