Last mile multihoming

Tue Mar 26 16:45:11 UTC 2013

On Mar 26, 2013, at 8:14 AM, Nick Zurku <Nick at nzurku.com> wrote:

> SOHO failover would be significantly easier if you had a VPN server in a
> datacenter, and setup something like pfSense to connect to the VPN over one
> or many ISP connections.

I'm essentially doing this now.

It does not reduce the DFZ impact. I advertise my routes out of the two
data centers where my VPNs terminate (well, GRE tunnels, really) and
run iBGP across the tunnels. It works just fine, but it's not simpler than
it would be if I could just peer with my direct upstreams.

> You really could just buy 2-3 local ISP connections, and let the VPN tunnel
> reestablish in the event of an outage (under a second, usually, states and
> connections preserved). I am unsure of bonding all those VPN connections at
> the same time, but I imagine there is a method to do that.

Yes, that's what happens today. How is managing a mesh of VPNs, equipment
in two additional data centers, a bunch of tunnels, and an extra 4+ BGP sessions
simpler?

Owen

> 
> On Mon, Mar 25, 2013 at 12:56 AM, Charles Wyble <
> charles-lists at knownelement.com> wrote:
> 
>> So isnt the most likely interruption to service due to a last mile
>> physical media issue?  Or say a regional fiber cut that takes out the
>> towers you can reach and the upstream connection from your cable and telco
>> providers? Imo at the edge, BGP mostly protects you from layer 8 fail  (if
>> youve done some basic best practice configuration). In theory, issues below
>> that (at least in the dist/core at l1 to 3) are handled by other redundancy
>> protections hidden from you (hsrp, fiber ring with protected path etc).
>> 
>> As for dfz explosion, would mpls/private as/ vrf be a workable approach
>> for bgp at the edge?
>> 
>> So I live in Austin. I have available to me two hfc providers (grande and
>> twc) and att. I also have sprint/clear vzw/tmo. I havent done an analysis
>> of wisp offerings (if any are on list, please email me at
>> charles at thefnf.org as im looking for a non ilec path for redunancy).
>> 
>> So lets break this down:
>> 
>> I only know of one att co in town. (Im sure if there is more, you will let
>> me know). So the chances of that failing are decently high. Also my
>> experience with att dsl have been mixed, unless im homed direct to the co.
>> Vz dsl otoh has always been rock solid. Also att is retiring dsl/copper. I
>> refuse to use uverse as they dont offer a unbundled modem/router or a way
>> to do bridge mode. Oh and no ipv6. (If you can put a modem in bridge mode
>> and still have working tv, please let me know. Ive not been able to find a
>> solution).
>> 
>> The chances of someone driving into the dslam serving my complex or the
>> pedastal down the street is high (100% as it has happend a couple times).
>> 
>> So this means I need a wireless backhaul. All of the providers I can reach
>> colocate on exactly one tower. Surrounded by a chain link fence, across
>> from a walmart. (Im in north austin near cameron and 183 for anyone who
>> lives in town). The chances of the fiber serving that tower being cut is
>> unknown, but not outside the realm of possibility. Or say the walmart big
>> rig over correcting due to a driver coming around the blind curve near
>> there and plowing into thr tower. Etc.
>> 
>> So my best bet for uninterrupted connectivity seems to be running two
>> openvpn tunels on my home edge pfsense router, each to a endpoint in a colo.
>> 
>> I already have a full rack of gear in joesdatacenter in kc, and its fully
>> redundant. I also run all of my web/mail/software dev from there, so its
>> not soley for bgp purposes. Most folks I imagine may have their stuff in a
>> colo as well and not want to run that at home. (I started a thread on that
>> once upon a time). It so happens, that I have various things which I cant
>> run there (rf equipment which I need to frequently reflash and move
>> around). So running bgp on my colo gear and announcing a /48 that ive
>> assigned to my house seems like a good idea. And I can easily cross connect
>> to kcix and have lots of bgp fun. The latency would be a bit high, but it
>> already is and I dont have any redundant connectivitym
>> 
>> Ok. So thats great. Now who is my secondary? Is a vps at say linode
>> sufficient for a secondary bgp announcer? Will they sell me bgp enabled
>> transit? Will other vps providers?  Do I need a box in a rack at a local
>> nap? Is there an ix in austin, or should I rack a box in Dallas?
>> 
>> Once i have two providerdls, then i can easily use pfsense multi wan
>> failover and if a circuit goes down, life goes on as I rely on bgp to
>> detect the link failure and handle it. Yes? No? Maybe?
>> 
>> So to me, this seems like a solved problem. Run multilple diverse
>> (carrier, media type) circuits to your edge, put a pfsense (asa, whatever
>> is your poison but i like pfsense the best for multi wan failover), openvpn
>> (i cant stand ipsec) to colo, cross connect to ... oh I dunno he.net :)
>> bgp for free. Done.
>> 
>> For about... hmmm.. 500.00 a month? (Many colos might not do bgp with you
>> for less then a quarter rack, and I presume anyone serious enough about
>> uninterrupted service on a reasonable budget can do 500.00 a month).
>> 
>> Thie discussion on soho multihoming has been fascinating to me, and I
>> wanted to go through a thought exercise for what I imagine is a common
>> scenario (main gear in a bgp enabled sp,  office gear needing to be
>> reachable by remote personnel in a non bgp enabled sp).
>> 
>> Would love to hear what you folks think.
>> 
>> 
>> 
>> --
>> Charles Wyble
>> charles at thefnf.org / 818 280 7059
>> CTO Free Network Foundation (www.thefnf.org)
>> 