gigabit router (was Re: Getting a "portable" /19 or /20)
Roeland Meyer
rmeyer at mhsc.com
Wed Apr 11 01:52:14 UTC 2001
> From: Craig Partridge [mailto:craig at aland.bbn.com]
> Sent: Tuesday, April 10, 2001 4:36 PM
>
> In message
> <Pine.BSF.4.21.0104101753540.98098-100000 at overlord.e-gerbil.net>, "R
> ichard A. Steenbergen" writes:
>
> >Don't be absurd, I can walk into fry's and pick up a motherboard with
> >64bit/66mhz PCI, some Netgear GA620's, and all the other
> >components for a 1GHz computer for under $1000.
>
> OK, so your bus has 4.2 Gb/s of bandwidth. But, alas, you're in a PC
> so you have to copy each packet from the line card, into main memory,
> examine it, and push it back out to a line card. So each
> packet consumes
> twice its size in bus bandwidth. So 2 1 Gb/s line cards will consume
> 4 Gb/s backplane. Assuming you can run the PCI at full rate (which in
> my experience is a big big if), you can connect two Ethernets.
There are a few server-class motherboards that have dual PCI-66 buses. One
on each bus and there's no problem. Incidently, those same MBs usually have
dual or quad CPUs.
> Incidentally, this isn't the full story either. You have to
> do a route lookup on each of those packets. That's typically 5 to 10
memory
> accesses... 5 memory access times 1 Mpps per gigabit times 2 gigabits
> is 10 million lookups per second or 100 ns per lookup. Allowing for
> time spent getting through the chip to the pins, you probably
> need 60 or 70ns DRAM, which is doable. Except, oops!, that completely
consumes
> your memory bandwidth... where are you going to find the cycles to
> get the packets in and out?
Huh?! We haven't had the 60ns limitation for over three years. What
time-vault have you been living in? PC100 RAM is error correcting at 10ns
and PC133 runs at ~8ns. In fact, I challenge you to show me where you can
even buy 60ns RAM these days. That's right up there with 3 GB HDDs, not
available because they can't find customers for any thing that small/slow,
except propriatary applications (they make you pay ... large). I just paid
$104 for 256MB of ECC PC100 RAM, at the local computer store. The discount
wasn't even worth the gas/hr to drive down to San Jose for (1989 Chevy
Suburban, in San Jose rush hour).
> PS: Side note, this illustrates where router vendors earn their bucks.
Router vendors earn their bucks by building routing/switching appliances.
The fact that they manage to get corps to cough up $200K for $15K worth of
hardware (COGm, generous) is yet another issue. I expect to see some serious
price drops in the next 6 months. I also expect to see commodity hardware
start making some serious in-roads (cause of the price-drop).
COGr = 1/2 COGm, manufacturers cost of raw material (chips, labor, copper,
and such)
COGm = Cost of Goods manufactured (fully burdened)
2*COGm = The resellers cost (including mfg's margins)
4*COGm = The distributors price
8*COGm = Retail price (What you pay, full retail)
[yeah, that's right. This means that, said RAM, cost $13 to make. econ 101]
> Find a way to move data over each bus only once (double your
> bandwidth!).
Dual PCI66 bus.
> Design your memory subsystems to keep packets and routing
> data separate (increase your memory bandwidth!).
PC133 ECC RAM (8ns).
> Find a processor that
> doesn't waste cycles doing virtual memory (improve your memory access
> times!).
Dual/Quad 1Ghz PIIIs, anyone?
> Oh yes, and then add hot board swap,
How many routers actually do this? Even when they do, it is more prudent to
find the time to shut it down for 5 minutes. It's not like solid-state
devices die all the time...no moving parts. Keep them cool and they'll go
obsolete before they fail. Unless it's a combat situation (where you don't
have time to swap anyway) there isn't much that warrants this level of
maintenance. You're much better off with a redundant hot fail-over
configuration so that you can bring one down for PM and replacement/upgrade.
> a working BGP implementation (quick, where's Tony Li working these
days:-)),
I actually like GateD. It ain't fancy, but it works.
> a CLI,
Linux is handy. RT Linux might be better (maybe).
> and a power subsystem for a CO,
If you must include that, then my choice is a Liebert GT3600.
> and you're in business.
With current commodity price drops, in less than 6 months, the cost of doing
this will fall well below $1000, retail. Considering potential market
volume. A 100% proprietary hardware solution may not be feasible, at that
price-point. They will *have* to use commodity hardware, just to keep up the
margins.
More information about the NANOG
mailing list