The Making of a Router

Sat Dec 28 15:45:24 UTC 2013

Chris Adams <cma at cmadams.net> wrote:
>Once upon a time, Shawn Wilson <ag4ve.us at gmail.com> said:
>> I was hoping someone could give technical insight into why this is
>good or not and not just "buy a box branded as a router because I said
>so or your business will fail". I'm all for hearing about the business
>theory of running an ISP (not my background or day job) but didn't
>think that's what the OP was asking about (and it didn't seem they were
>taking business suggestions very well anyway).
>
>There's been some technical insight here I would say.  I'm a big Linux,
>Open Source, and Free Software advocate, and I'll use Linux-based
>systems for routing/firewalling small stuff, but for high speed/PPS,
>get
>a router with a hardware forwarding system (I like Juniper myself).
>
>You can build a decently-fast Linux (or *BSD) system, but you'll need
>to
>spend a good bit of time carefully choosing motherboards, cards, etc.
>to
>maximize packet handling, possibly buying multiple of each to find the
>best working combination.  Make sure you buy a full set of spares once
>you find a working combination (because in the PC industry, six months
>is a lifetime).  Then you have to build your OS install, tweaking the
>setup, network stack, etc.
>
>After that, you have to stay on top of updates and such (so plan for
>more reboots); while on a hardware-forwarding router you can mostly
>partition off the control plane, on a Linux/*BSD system, the base OS is
>the forwarding plane.  Also, if something breaks, falls over under an
>attack, etc., you're generally going to be on your own to figure it
>out.
>Maybe you can Google the answer (and hope it isn't "that'll be fixed in
>kernel 3.<today's version+2>.  Not saying that doesn't happen with
>router vendors (quoting RFCs at router engineers is "fun"), but it is
>IMHO less often.
>
>The question becomes: what is your time worth?  You could spend
>hundreds
>of hours going from the start to your satisfactory in-service router,
>and have a potentially higher upkeep cost.  Can you hire somebody with
>all the same Linux/*BSD knowlege as yourself, so you are not on-call
>for
>your home-built router around the clock?
>
>I've used Linux on all my computers for almost 20 years, I develop on
>Linux, and contribute to a Linux distribution.  However, when I want to
>record TV to watch later, I plug in a TiVo, not build a MythTV box.
>There is a significant value in "just plug it in and it works", and if
>you don't figure your time investment (both up-front and on-going) into
>the cost, you are greatly fooling yourself.

I agree with all of this to some degree. IDK whether cost of ownership on a hardware router or a desktop is more or less - I jus haven't done the research. We use them at work and at home I have Cisco and Linksys gear (plus Linux doing some things the router could like DHCP) - go figure.

I agree that some network cards and boards work better than others (and am partial to the Intel Pro cards - though I'm unsure if they're still the best). I would also hesitate to route that much traffic with a PC. Though, I have no technical reason for this bias. 

If you have hardware in production, you really should have a spare - whether we're talking servers, HDDs, batteries, or routers. Ie, that comment is not unique to servers. I also don't think warranty has any bearing on this - I've seen servers stay down for over a day because (both HP and Dell for their respective hardware) screwed up and the company didn't budget for a spare board and I've seen a third of a network be taken out because multiple switch ports just died. How much would a spare switch have cost compared to 50 people not online?

At any rate, I'm interested in this because I've worked in both environments and haven't seen a large difference between the two approaches (never worked at an ISP or high bandwidth web environment though). I do like the PC router approach because it allows more versatility wrt dumping packets (no need to dig out that 10mbit dumb hub and throttle the whole network), I can run snort or do simple packet inspection with iptables (some routers can do this but most can't or require a license). So I'm sorta leaning to the PC router as being better - maybe not cheaper but better. 