The Making of a Router
symack at gmail.com
Thu Dec 26 21:22:18 UTC 2013
Inline response exist,
On 12/26/13, Ray Soucy <rps at maine.edu> wrote:
> You can build using commodity hardware and get pretty good results.
> I've had really good luck with Supermicro whitebox hardware, and
> Intel-based network cards. The "Hot Lava Systems" cards have a nice
> selection for a decent price if you're looking for SFP and SFP+ cards that
> use Intel chipsets.
I like the supermicro as well however we have a couple of IBM x3250
with 2 pcie v3
x8 that are begging for a intel network card.
> There might be some benefits in going with something like FreeBSD, but I
> find that Linux has a lot more eyeballs on it making it much easier to
> develop for, troubleshoot, and support. There are a few options if you
> want to go the Linux route.
This is very important to consider. I would be speculating, or even
the same type of community support from the BSD verse that I have been
getting from the linux community.
> Option 1: Roll your own OS. This takes quite a bit of effort, but if you
> have the tallant to do it you can generally get exactly what you want.
If Free/OpenBSD is ruled out, I could crack open the LFS project. You only
have to do it once right? Or maybe just reach out to the gentoo community
for a stripped version, and build outwards.
> The biggest point of failure I've experienced with Linux-based routers on
> whitebox hardware has been HDD failure. Other than that, the 100+ units
> I've had deployed over the past 3+ years have been pretty much flawless.
> Thankfully, they currently run an in-memory OS, so a disk failure only
> affects logging.
> If you want to build your own OS, I'll shamelessly plug a side project of
> mine: RAMBOOT
> RAMBOOT makes use of the Ubuntu Core rootfs, and a modified boot process
> (added into initramfs tools, so kernel updates generate the right kernel
> automatically). Essentially, I use a kernel ramdisk instead of an HDD for
> the root filesystem and "/" is mounted on "/dev/ram1".
> The bootflash can be removed while the system is running as it's only
> mounted to save system configuration or update the OS.
> I haven't polished it up much, but there is enough there to get going
> pretty quickly.
Ummm, if it's ok with the community, can you kindly elaborate :). I am
not too fond of Debian since my horrible experience with Squeeze Desktop.
I would maybe like to try this using the combination of SSD, in memory, and
> You'll also want to pay attention to the settings you use for the kernel.
> Linux is tuned as a desktop or server, not a router, so there are some
> basics you should take care of (like disabling ICMP redirects, increasing
> the ARP table size, etc).
Totally strip it as much as possible. If anyone has a Gentoo stripped kernel
config that they would like to share, please do :).
> I have some examples in: http://soucy.org/xorp/xorp-1.7-pre/TUNING
> or http://soucy.org/tmp/netfilter.txt (more recent, but includes firewall
Will definitely look into all your sites.
> Also a note of caution. I would stick with a longterm release of Linux.
> I've had good experience with 2.6.32, and 3.10. I'm eager to use some of
> the post-3.10 features, though, so I'm anxious for the next longterm branch
> to be locked in.
We are comfy with 3.4 right now...
> One of the biggest advantages is the low cost of hardware allows you to
> maintain spare systems, reducing the time to service restoration in the
> event of failure. Dependability-wise, I feel that whitebox Linux systems
> are pretty much at Cisco levels these days, especially if running
Really interested with the "in-memory", however, I would love to implement
it using gentoo as mentioned above.
More information about the NANOG