The Making of a Router

Thu Dec 26 21:22:18 UTC 2013

Inline response exist,

On 12/26/13, Ray Soucy <rps at maine.edu> wrote:
> You can build using commodity hardware and get pretty good results.
>
> I've had really good luck with Supermicro whitebox hardware, and
> Intel-based network cards.  The "Hot Lava Systems" cards have a nice
> selection for a decent price if you're looking for SFP and SFP+ cards that
> use Intel chipsets.

I like the supermicro as well however we have a couple of IBM x3250
with 2 pcie v3
x8 that are begging for a intel network card.

> There might be some benefits in going with something like FreeBSD, but I
> find that Linux has a lot more eyeballs on it making it much easier to
> develop for, troubleshoot, and support.  There are a few options if you
> want to go the Linux route.

This is very important to consider. I would be speculating, or even
worse, expecting
the same type of community support from the BSD verse that I have been
getting from the linux community.

>
> Option 1: Roll your own OS.  This takes quite a bit of effort, but if you
> have the tallant to do it you can generally get exactly what you want.

If Free/OpenBSD is ruled out, I could crack open the LFS project. You only
have to do it once right? Or maybe just reach out to the gentoo community
for a stripped version, and build outwards.

> The biggest point of failure I've experienced with Linux-based routers on
> whitebox hardware has been HDD failure.  Other than that, the 100+ units
> I've had deployed over the past 3+ years have been pretty much flawless.
>

SSD

> Thankfully, they currently run an in-memory OS, so a disk failure only
> affects logging.
> If you want to build your own OS, I'll shamelessly plug a side project of
> mine: RAMBOOT
>
> http://ramboot.org/
>
> RAMBOOT makes use of the Ubuntu Core rootfs, and a modified boot process
> (added into initramfs tools, so kernel updates generate the right kernel
> automatically).  Essentially, I use a kernel ramdisk instead of an HDD for
> the root filesystem and "/" is mounted on "/dev/ram1".
>
> The bootflash can be removed while the system is running as it's only
> mounted to save system configuration or update the OS.
>
> I haven't polished it up much, but there is enough there to get going
> pretty quickly.

Ummm, if it's ok with the community, can you kindly elaborate :). I am
not too fond of Debian since my horrible experience with Squeeze Desktop.
I would maybe like to try this using the combination of SSD, in memory, and
Gentoo?

>
> You'll also want to pay attention to the settings you use for the kernel.
>  Linux is tuned as a desktop or server, not a router, so there are some
> basics you should take care of (like disabling ICMP redirects, increasing
> the ARP table size, etc).

Totally strip it as much as possible. If anyone has a Gentoo stripped kernel
config that they would like to share, please do :).

>
> I have some examples in: http://soucy.org/xorp/xorp-1.7-pre/TUNING
> or http://soucy.org/tmp/netfilter.txt (more recent, but includes firewall
> examples).

Will definitely look into all your sites.

>
> Also a note of caution.  I would stick with a longterm release of Linux.
>  I've had good experience with 2.6.32, and 3.10.  I'm eager to use some of
> the post-3.10 features, though, so I'm anxious for the next longterm branch
> to be locked in.
>

We are comfy with 3.4 right now...

> One of the biggest advantages is the low cost of hardware allows you to
> maintain spare systems, reducing the time to service restoration in the
> event of failure.  Dependability-wise, I feel that whitebox Linux systems
> are pretty much at Cisco levels these days, especially if running
> in-memory.

Really interested with the "in-memory", however, I would love to implement
it using gentoo as mentioned above.

Kind Regards,

N.