Enterprise Multihoming

Fri Mar 12 17:18:08 UTC 2004

At 4:06 PM +0000 3/12/04, Stephen J. Wilcox wrote:
>I think its too easy, thats the problem.

Hoping that I don't sound too much like Bill Clinton, that depends on 
what you mean by "it." If "it" is multihoming, with your own ASN, to 
two providers, your raise some valid points.

Is there an intermediate alternative before you go all out?  Yes, I 
think so, assuming your current provider has multiple POPs.  Let me 
examine some of your points if we consider RFC 1998-style 
multi-POPping (I just invented that highly technical term) using PA 
address space.

>For <$1000 (excluding bandwidth/ccts)
>you can buy a box, connect to your two providers, get an ASN and IPs 
>and you're
>away.

Alternatively, another POP link, and preferably another router. If 
you are more concerned with loop failures than router failures, not a 
completely unreasonable assumption, you could get away with one 
router that has multiple interfaces, and spend some of the savings on 
backup power -- possibly a backup power supply in addition to the 
UPS, such as a Cisco RPS on their smaller routers.  While you'll 
probably take a performance hit, or if you can reduce to critical 
traffic on an outage, you might get away with a second smaller router.
>
>I dont agree that connecting to two+ upstreams makes you better. In my
>experience end networks have a couple of orders of magnitude more 
>downtime than
>a PoP in any reasonably large ISP. Ie the percentage theoretical 
>improvement is
>small.

Like everything else, It Depends. My experience is that access links 
fail more often than provider routing systems, especially with a 
clueful provider.  Since you can't guarantee that your physical 
connectivity to two different ISPs doesn't involve a shared risk 
group in the lines, there are still some things you may not be 
protected against.

One option, depending on the plant in your area, is that if you are 
considering a second router, consider putting it in a nearby 
building, reachable by WLAN (if you are minimizing costs), where that 
building minimally has different ducts to the telco end office, and 
ideally goes to a different end office. Not always possible, but to 
be considered.  Longer-range wireless (radio or optical) links get 
more expensive.

>
>In addition you seriously increase the complexity of your system, chances are
>you're using the cheapest kit you could find (or at least cheaper and smaller
>than what I would use).. its not great at BGP and may fall over when you get a
>minor DoS attack, you probably generate flaps quite a bit from adhoc 
>changes and
>if you're announcing a /24 then thats going to get you dampened quickly..

That's a motivation for PA address space, where the provider 
aggregate is less likely to be small and easily damped.

>  so you
>actually create a new weakest link. Also most of the corporates I've 
>dealt with
>take defaults rather than full tables.. so if the provider does have an issue
>you still forward the traffic, theres no failover of outbound routing.

Again looking at intermediate solutions, there are always partial 
routes such as customer routes of the provier.

>
>Even if you spend (waste) the money on some decent gear, you're on 
>your own and
>when a problem occurs the ISPs are going to be less helpful to you (not by
>choice, I mean they dont have control of your network any more.. 
>there knowledge
>of whats causing problems is limited to the bit that they provide to you), so
>chances are your problems may be more serious and take longer to diagnose and
>fix.

Again, an operational advantage of multiPOPping and working with one 
carrier, although you aren't going to be protected against insanity 
of their BGP/

>
>IMHO avoid multihoming. You will know when you are big enough and 
>you *need* to
>do it, if you're not sure or you only want to do it cause you heard everyone
>else is and its real cool then I suggest you dont.

MHO would be to look at "multihoming" as a spectrum of solutions 
rather than a binary choice of single-provider-single-link versus 
multiple-provider.  In given situations, you might also want to look 
at DSL or cable for diversity, tunneling to an ISP since the 
broadband provider is unlikely to be willing to speak BGP. Even 
dialup/ISDN, sometimes for critical workstations, has its place.

Shameless plug:  I do go through these options in my book, Building 
Service Provider Networks (Wiley).  Even there, though, I only run 
through the alternatives. You will still have to make your own 
cost-benefit decisions based on business policy, budget, clue level 
and cost of alternatives.