BGP peering strategies for smaller routers

Tue May 3 14:31:15 UTC 2016

On Mon, May 2, 2016 at 3:07 PM, Mike <mike-nanog at tiedyenetworks.com> wrote:
>     I have an ASR1000 router with 4gb of ram. The specs say I can get '1
> million routes' on it, but as far as I have been advised, a full table of
> internet routes numbers more than 530k by itself, so taking 2 full tables
> seems to be out of the question (?).

Hi Mike,

There is no "routing table." Instead:

Routing Information Bases (RIB)
Forwarding Information Base (FIB)

RIB = the routing data from your neighbors. Each protocol (e.g. BGP,
OSPF, static, connected) has a RIB. If you have virtual routers (VRF)
then each VRF has its own RIBs. For the BGP RIB, each prefix (route)
from each neighbor will need its own slot in the RIB.

FIB = the next hop table. Only the best next hop for each prefix is
stored in the FIB. One FIB per protocol (IPv4, IPv6), unless you run
VRFs, then one FIB each per VRF.

All routers have both a FIB and RIBs. All routers keep the RIBs in
DRAM. Big Iron like yours typically store the FIB in special hardware
called Ternary Content Addressable Memory (TCAM). You're looking at
two physically different kinds of memory in your router to store the
RIB and the FIB. No matter how much DRAM you have, you only have so
much TCAM in that routing engine. The TCAM is not upgradable. In fact,
it's buried under a big heat sink.

All the RIBs are processed down to a single FIB for each protocol/VRF.
The best next hop is selected from all the possibilities in the RIBs
and only that one gets stored in the FIB. The FIB in the TCAM is
consulted every single time a packet is routed. The RIBs are only
consulted when new information is received from a neighbor of when the
FIB table needs to be rebuilt. The RIB is not consulted to route
individual packets.

Your "million routes" is a reflection of the TCAM on your routing
engine. The FIB table size. 4G DRAM is enough for 10-15 million routes
in the various RIBs. If you're running a single VRF, you have enough
FIB headroom for the next two or three years, until the v4 and v6 BGP
tables add up to around 900M prefixes. You already have too little FIB
headroom to run two BGP-speaking VRFs.

My piddly little 2811 carries a full IPv4 table, 580k routes. Its BGP
RIB consumes all of 154 megabytes of DRAM. Unlike your router, the
2811 does not have a "hardware fast path." No TCAM. It stores the FIB
in a radix tree in DRAM instead. As a result, it can only handle low
data rates, in the 10s of megabits per second.

I'm leaving out lots of details, but these are the most important with
respect to your question.

Regards,
Bill Herrin

-- 
William Herrin ................ herrin at dirtside.com  bill at herrin.us
Owner, Dirtside Systems ......... Web: <http://www.dirtside.com/>