What is the limit? (was RE: multi-homing fixes)

Thu Aug 30 23:12:05 UTC 2001

|> From: Sam Thomas [mailto:sthomas at lart.net]
|> Sent: Thursday, August 30, 2001 2:57 AM
|> 
|> On Wed, Aug 29, 2001 at 11:36:03PM -0700, Roeland Meyer wrote:
|> > 
|> > |> From: Randy Bush [mailto:randy at psg.com]
|> > |> Sent: Wednesday, August 29, 2001 9:18 PM
|> > |> 
|> > |> > there would appear to be people who assert its not 
|> > |> > unafordable to compute the routes for the current,
|> > |> > and the forseeable network.
|> > |> 
|> > |> and there are those of us who think we can probably do 
|> > |> so iff we stop the explosion of /24s.
|> > 
|> > Okay, let's get it back out of the realm of opinion and 
|> > into requirements.
|> > I'll even go down one step more, to /25. Take the entire 
|> > IP addr space,
|> > break it ALL into /25's, times the BGP table entry size, 
|> > how big is that?

|> > Also, I just got a price quote for 512MB ECC PC133 DIMMS 
|> > at $49US, retail.

|> using Jonthan Disher's calculation of 155B/route (a check 
|> of my local neighborhood gsr gave ~185B/route, but this is 
|> all academic anyway so we'll use his number since it was 
|> first and so folk playing along at home can have some 
|> continuity), i get ~5GB for the whole address space as /25s.
|> (using 185B/route, it came to just over 8GB). 

Thank you, it was the route-entry size that I didn't have defensible numbers
on. I knew roughly what they were, but was already out on the limb.<g>

|> i don't know of any off the shelf consumer hardware that 
|> supports >2GB (i looked, really!), so the price to stick 
|> that $800 ($16 is extremely reasonable shipping cost for 
|> 16 memory sticks, see if you can find it ;) of ram on 
|> something just went up an order of magnitude.

Okay, well I wasn't talking about COTS boards anyway. I was talking about
commodity tech, at the chip level. What us old microP silicon mechanics
called the "component" level. Circuit boards all cost the same to build and
design, regardless of the content of the design, with minor variations. It's
ALL automated these days. Feed a schematic in one end and, about a week
later, get a circuit board design out the other, with none-to-little human
intervention. Manufacturing is about the same. It is the components that
attach to them that contains the variable cost. It is those components whose
cost is dropping like lemmings off a cliff. You see, chip design and
manufacture is almost getting to the same place as circuit boards. The only
human-introduced variable is in the original design and architecture. That
cost is amortised over the volume of units sold. This is where commodity
tech has a huge cost/performance advantage.

|> > Considering what router vendors charge, with very fat 
|> > margins, for their low-tech routers, affordable tech 
|> > arguments don't hold water. Routers are expensive 
|> > because someone is stupid enough to pay the prices. 

|> i guess we're in the wrong business. we should go develop 
|> discount routers using near-current off-the-shelf 
|> technology. thinking this through, i'm already amazed at 
|> the lack of glut for discount high-capacity routers.

You haven't been paying attention to the stock market then. Why do you think
router vendor stock prices are dropping like flies? Maybe because there *is*
a glut of cheap used equipment? This is forcing more sane prices on them
and, as a direct result, their earnings reports are down.

|> oops. some folk (like me) actually want the core router 
|> to be able to handle a chassis chock-full-o-oc48s. 
|> that'll take a bit of specialized hardware. oh, and 
|> it would be really nice-ah if the router could forward
|> ip packets amongst those oc48s at something approaching 
|> line-rate. that'll take an industrial strength backplane. 
|> not finding too many multi-Gb/s backplanes on pricewatch.com, 
|> so guess that will have to be custom built, and we'll 
|> probably have to develop some custom circuitry to handle the
|> custom backplane as well.

I'll bet that COGm, for a Cisco Catalyst 6509, with all GigE blades, is
under $5KUS. Further, that the major cost is the sheet-metal and paint work,
not the electronix. In the last three years, I emplaced 6 of them at ~$200K
each. Please tell me that isn't a very fat margin and yes, I do know about
keystone pricing and how the distro channel works. Even R&D isn't a major
cost, given enough volume. Manufacturing has been commoditized for over 20
years. Are you telling me that Cisco can't give us better tech, for less
money, than they've been doing thus far?

|> shoot, forgot software. we've run up the cost so much, we'll 
|> just drop in a peecee with a linux kernel and zebra. we 
|> won't be able to route packets through it, though, so i 
|> guess we'll have to custom manufacture some other
|> device to handle the actual packing shuffling. or, we 
|> could custom develop a single unit and some software that 
|> runs on it. i'm sure there are plenty of 
|> bgp/mpls/multicast/etc clued programmers willing to work
|> in our software sweatshop for minimum wage to keep costs low.

see, Richard Stallman and the FSF. I'll save my thoughts on open-source
"free" software for another day. Please know that MHSC is a Caldera VAR
(still dealing with Caldera assilimating the "enemy", SCO), and we can sell
you a copy of SCO OpenUNIX these days (but not cheap <sigh>). Yes, I've had
GateD running for some time now. Software cost is a part of R&D costs.

|> so, after all this, how big of a bargain is using that $800 
|> in ram so far?

All the additional conditions should make ZERO difference to the COGs of the
product. Are you telling me that 8GB worth of table entries is a small
market?

|> ok, it's still probably at least a few bucks cheaper than 
|> vendor X. let's go sell it. hah! silly customers want a 
|> number to call when it breaks. well, it uses cheap 
|> off-the-shelf memory and processor, how much can it really
|> cost to support? 

You know ... in the past 10 years, with many hundreds of servers, I have had
ZERO CPU failures and ZERO RAM failures, past the infant mortality period.
Even in workstations, all such failures were caused by brilliant actions,
like restricting the airflow (wrapping the darned thing in a blanket, once),
even then, the HDDs were the first to die. With zero moving parts, if you
can keep them cool, they run forever. They obsolece long before they die. I
have one old server, an amdK6-233, whose RAM (128MB of SIMMs) has been in
24x7 operation for over 6 years. It's also running a AHA2940 that has seen
24x7 operation since 1996.

|> since we used commodity hardware (at least ram and 
|> processor), we won't have to stock spares. i'm sure
|> we can trust our customers to buy the right parts 
|> off the shelf at "Bubba's Fine House of PeeCee Hardware".

You can kid around if you want to. But, you're only showing irrational bias.
An Intel part is an Intel part, regardless of whether Bubba or Einstein
sells it to you. Likewise with RAM. If the board don't take standard PC133
RAM then the design is defective. Bubba doesn't make the parts. He only
sells them. Unless, Bubba has a silicon foundry in his basement. In that
case, his name probably isn't Bubba.

|> now, would everyone kindly get off the $EXPENSIVE ram from 
|> vendor X whine?

Those who discount, discounts, wind up broke, before those that don't.

|> > royally ripped off there. Cisco takes special RAM because 
|> > Cisco designed incompatible RAM circuitry so they could 
|> > charge you more for the Cisco label. It also bolsters 
|> > the FUD-storm and the after-market support vendors.
|> 
|> or, maybe it's because they had the foresight to know that 
|> if they used commodity ram, a bunch of cheapskate morons 
|> would be plugging in the cheapest ram they could find at 
|> the saturday morning swapmeet, and making dozens of
|> "warranty" support calls because the router crashes all the 
|> time or half the packets come through looking like 
|> swiss cheese.

Maybe the alternative was to hire competent circuit design engineers that
could design circuits that don't have those vulnerabilities? In my book, the
moron was the fool that didn't design with those cheapskates in mind.