SP / Enterprise design (dis)similarities
keegan.holley at sungard.com
Tue Oct 11 00:12:13 CDT 2011
2011/10/10 Tom Lanyon <tom+nanog at oneshoeco.com>
> Hi all,
> Looking for some advice or experience in a small enterprise / hosting
> provider context.
> There's plenty of BCP information around for SPs in the network design
> realm, and I'm curious how much of this applies to enterprises too.
> Commonly advised items like:
> * pull-up statics created on core devices, not network border
> * using iBGP to carry customer prefixes, not an IGP
> * announcing defaults over iBGP or IGP
> It depends on the size of the enterprise as well as the service provider.
> In some cases I imagine it may be simpler to have all BGP finish at the
> network border devices and not have to worry about running both IGP and iBGP
> sessions inwards to the core and/or aggregation devices.
In alot of cases iBGP is easier than an IGP if you have a large number of
routes or a large number of routers. In others it's easier to accept only a
default from the SP and maybe a local table. This is highly subjective as
> I understand the limitations of putting our Internet prefixes in an IGP,
> but for a hosting provider style network where everything is ethernet
> connected and within data centres there's much less route flapping to deal
> with (there's no bouncing DSL lines, for example).
This depends as well. There are pleanty of hosting customers that want a
full or partial table which means you'll have to run BGP. There are all
sorts of other limitations with and IGP as well.
> In the case that there is both iBGP and IGP running internally, is there
> any reason to choose one or the other to originate a default route to our
> aggregation/access layers? At some point I imagine it's going to be
> redistributed into the IGP (or re-originated in the IGP), so would think it
> would be best to just always run the default in the IGP to keep things
The more route aggregation the larger the potential for loops. Then again
this is subjective as well. In general BGP isn't something to be avoided at
all costs even at a small scale. It's much easier to scale a topology that
is iBGP based than to try to move to an iBGP setup after it has grow too
large for the IGP.
> Finally - are there any reasons to avoid running next-hop-self on ibgp
I would go the other way and ask if there is any reason to advertise all the
next-hop address into the IGP?
> The upside is we get to avoid distributing all of our transit/peer upstream
> point to point links into the rest of the network. Again, I understand this
> may be undesirable from a SP perspective, but when our 'clients' are all a
> bunch of internal servers it makes sense to keep iBGP/IGP as clean as
The definition of clean is also subjective. There are many who would run
the IGP only for loopbacks and /30's and force everything into BGP even at
small scale. BGP makes it easier to control the routing relationships
between companies and pretty much removes the need for redistribution.
There are trade-offs though, such as load-balancing.
It really depends on your environment though and the preference of your
engineering team. In general there isn't really a such thing as SP vs.
Enterprise. For example I've know of a large bank with an internal MPLS
topology and about 8 public AS's used just to segregate traffic by country
of origin. By most definitions this is an enterprise network although a
very large one. On the other hand I've seen a service provider who's
network consisted of 4 M10i's and a firewall in someone else's colo
facility. You have to look at which protocols and services you need to make
your network work as well as the ability to scale. That should dictate your
design more than an imaginary line between SP and enterprise.
More information about the NANOG