gigabit router (was Re: Getting a "portable" /19 or /20)

Craig Partridge craig at aland.bbn.com
Thu Apr 12 14:16:00 UTC 2001



In message <20010412100249.B15380 at alcor.net>, Matt Zimmerman writes:

>It _does_ show that distributed forwarding will be significantly more complex
>and more difficult to implement.  I am not arguing that distributed forwarding
>is a bad thing; on the contrary, it has a logical and demonstrated performance
>benefit.  However, it comes with a cost in overall system complexity.

Having implemented distributed forwarding once, and consulted on it a
couple of times, the answer is that it adds complexity but not where
you'd think.

You usually only have to generate one forwarding table (the outbound
interface through which you send the packet doesn't change depending on
whether you make the decision centrally or at the inbound interfaces).
You can try to reduce the size of the forwarding table by excising stuff
(e.g., info on VPNs that don't run through that interface) but forwarding
tables are pretty small so you don't have to do that if you don't want to.

Provided you don't do something stupid and have the outbound interface
re-evaluate the inbound interface's routing decision, then all you need
is a way to copy the forwarding table to each interface (and if the
interfaces receive the table at slightly staggered times that's OK,
because they don't have to be perfectly consistent).

Now I just said "all you need" -- that's a lie :-).  The major complexity is
actually getting the forwarding tables out to the inbound interfaces.
There are at least two issues: (a) bandwidth [a small forwarding table,
times 32 or 64 ports rapidly becomes big, especially for the control
data path which is typically short on bandwidth] and (b) finding a way to
update the forwarding table in a way that is not too disruptive to forwarding
[incremental updates are often mindbreakingly hard to do, esp. if the
update changes the table size, or invalidates some entries -- you can
have the forwarding table doing housecleaning instead of forwarding].
We used a bank swap -- we had an active forwarding table and an inactive
one.  Updates were sent to the inactive bank and when you had a fully
updated bank, you send a message to the forwarding processor saying
"swap banks."  And, of course, the software to track which forwarding
engine had which bank live and the current state of each bank's updates
got a little exciting.

In short, generating the tables is easy, consistency isn't an issue,
but getting the tables to the right place, in a timely manner, and without
getting confused and trashing a table, is a challenge.

Thanks!

Craig




More information about the NANOG mailing list