The Making of a Router

Blake Dunlap ikiris at
Sat Dec 28 19:50:35 UTC 2013

Pretty much what everyone else said. I'm a huge linux person, almost
everything I use is linux, run full Myth set up etc, but I wouldn't use it
for a high PPS situation like this. It's just asking for suffering later,
at the worst possible times.


On Sat, Dec 28, 2013 at 9:45 AM, Shawn Wilson < at> wrote:

> Chris Adams <cma at> wrote:
> >Once upon a time, Shawn Wilson < at> said:
> >> I was hoping someone could give technical insight into why this is
> >good or not and not just "buy a box branded as a router because I said
> >so or your business will fail". I'm all for hearing about the business
> >theory of running an ISP (not my background or day job) but didn't
> >think that's what the OP was asking about (and it didn't seem they were
> >taking business suggestions very well anyway).
> >
> >There's been some technical insight here I would say.  I'm a big Linux,
> >Open Source, and Free Software advocate, and I'll use Linux-based
> >systems for routing/firewalling small stuff, but for high speed/PPS,
> >get
> >a router with a hardware forwarding system (I like Juniper myself).
> >
> >You can build a decently-fast Linux (or *BSD) system, but you'll need
> >to
> >spend a good bit of time carefully choosing motherboards, cards, etc.
> >to
> >maximize packet handling, possibly buying multiple of each to find the
> >best working combination.  Make sure you buy a full set of spares once
> >you find a working combination (because in the PC industry, six months
> >is a lifetime).  Then you have to build your OS install, tweaking the
> >setup, network stack, etc.
> >
> >After that, you have to stay on top of updates and such (so plan for
> >more reboots); while on a hardware-forwarding router you can mostly
> >partition off the control plane, on a Linux/*BSD system, the base OS is
> >the forwarding plane.  Also, if something breaks, falls over under an
> >attack, etc., you're generally going to be on your own to figure it
> >out.
> >Maybe you can Google the answer (and hope it isn't "that'll be fixed in
> >kernel 3.<today's version+2>.  Not saying that doesn't happen with
> >router vendors (quoting RFCs at router engineers is "fun"), but it is
> >IMHO less often.
> >
> >The question becomes: what is your time worth?  You could spend
> >hundreds
> >of hours going from the start to your satisfactory in-service router,
> >and have a potentially higher upkeep cost.  Can you hire somebody with
> >all the same Linux/*BSD knowlege as yourself, so you are not on-call
> >for
> >your home-built router around the clock?
> >
> >I've used Linux on all my computers for almost 20 years, I develop on
> >Linux, and contribute to a Linux distribution.  However, when I want to
> >record TV to watch later, I plug in a TiVo, not build a MythTV box.
> >There is a significant value in "just plug it in and it works", and if
> >you don't figure your time investment (both up-front and on-going) into
> >the cost, you are greatly fooling yourself.
> I agree with all of this to some degree. IDK whether cost of ownership on
> a hardware router or a desktop is more or less - I jus haven't done the
> research. We use them at work and at home I have Cisco and Linksys gear
> (plus Linux doing some things the router could like DHCP) - go figure.
> I agree that some network cards and boards work better than others (and am
> partial to the Intel Pro cards - though I'm unsure if they're still the
> best). I would also hesitate to route that much traffic with a PC. Though,
> I have no technical reason for this bias.
> If you have hardware in production, you really should have a spare -
> whether we're talking servers, HDDs, batteries, or routers. Ie, that
> comment is not unique to servers. I also don't think warranty has any
> bearing on this - I've seen servers stay down for over a day because (both
> HP and Dell for their respective hardware) screwed up and the company
> didn't budget for a spare board and I've seen a third of a network be taken
> out because multiple switch ports just died. How much would a spare switch
> have cost compared to 50 people not online?
> At any rate, I'm interested in this because I've worked in both
> environments and haven't seen a large difference between the two approaches
> (never worked at an ISP or high bandwidth web environment though). I do
> like the PC router approach because it allows more versatility wrt dumping
> packets (no need to dig out that 10mbit dumb hub and throttle the whole
> network), I can run snort or do simple packet inspection with iptables
> (some routers can do this but most can't or require a license). So I'm
> sorta leaning to the PC router as being better - maybe not cheaper but
> better.

More information about the NANOG mailing list