Multi-POP design check/help question

Sun Sep 20 20:17:41 UTC 2009

Cross-posted from cisco-nsp.  We are a (mostly) Cisco shop, but I'm looking
more for BCP and overall design, not provisioning specifics.

-----

My Cisco bookshelf isn't helping me much with this...

We currently have a single POP with border/core/aggregation topology.
Upstreams each come in on their own border router and the core is used as a
route-reflector for border and aggregation. The internal network uses OSPF
as an IGP and each device is dual-connected for redundancy on independent
layer-2 networks.  OSPF load-shares with loopback IPs and IBGP uses the
loopback addresses for peering.

We are looking at turning up two additional POPs in the metro area, each
connected by redundant GigE loops to the original POP.  Each POP may have
zero or more direct upstream connections.  I'd like local traffic at each
POP to prefer both in and outbound traffic via the local upstream, but still
be able to failover to upstreams at other POPs if needed.

My initial thoughts are to BGP peer between POPs with a higher local-pref
for the local outbound traffic and to prepend between the POPs so inbound
traffic is more likely to take the shortest path inbound.

Is this too simplistic? Prone to trouble? What gotchas should I be looking
at, or other designs should I be considering?