Failover how much complexity will it add?

Blake Pfankuch bpfankuch at
Sun Nov 8 15:23:41 UTC 2009

>> -----Original Message-----
>> From: adel at [mailto:adel at]
>> Sent: Sunday, November 08, 2009 4:52 AM
>> To: nanog at
>> Subject: Failover how much complexity will it add?
>> HI,
>> I was recently brought onto a project where some failover is desired, but I think that the number of connections provisioned
>> is excessive.  Also hoping to get some guidance with regards to how well I can get the failover to actually work.  So currently
>> 4 X 100Mb/s Internet connections have been provisioned.  One is to be used for general Internet, out of the organisation, it
>> also terminates VPNs from remote sites belonging to the organisation and some publicly accessible servers -routed DMZ and
>> translated IPs.  Second Internet connection to be used for a separate system which has a site-to-site VPN to a third party
>> support vendor.  Internet connections 3 and 4 are currently thought of as providing backups for one and two.  Both connections
>> firewalled by a Juniper SSG of some description.
>> Now I couldn't get any good answers as to why Internet connections 1 and 2 need to be separate.  I think the idea was to make
>> sure that there was enough bandwidth for the third party support VPN.  I feel that I can consolidate this into one connection
>> and just use rate limiting to reserve some portion of the bandwidth on the connection and this should be fine.  Now if I was to
>> do this then I can make a case for just having one backup Internet connection.  However I'm still concerned about failover and
>> reliability issues.  So my questions regarding this are:
>> - Should I make sure that the backup Internet connection is from a separate provider?

Yes yes yes yes a thousand times yes.  Depending on the criticality of internet connectivity you should also aim to have your redundant
connections coming from a complete separate direction.  Example, fiber from Level 3 come from the north in a dedicated conduit and
fiber from Verizon coming in a dedicated conduit from the south of the building.  Why?  Put simply we had construction ignore the
painted lines and dig up our conduit a few years back.  At that point we have 4 bonded T1's from a single carrier.  That was a long
couple of days...  Carrier diversity is not a bad thing, spend some time shopping an additional provider.  Make sure they operate their
own network for last mile, and also make sure they don’t piggyback off the same network your main carrier does anywhere locally.
Comcast Ethernet, Verizon and Cogent make great secondary connections when you need high availability.  You don’t need your
secondary to have 99.999% uptime.  97% is usually good enough if it's on a separate network.  I wouldn't sway from the big names
for your primary connections either.

>> - How can I acheive a failover which doesn't require me to change all the remote VPN endpoints in case of a failover?  Its
>> possible to configure failover VPNs on the Junipers, which should take care of this, but how do I take care of the DMZ hosts and
>> external translation?

With recent experience with the Juniper SSG VPN functions put nicely they suck.  VPN failover is in there, but we had issues with the
tunnel staying active for extended periods of time.  Also depending on if you do a route based or a policy based VPN, it becomes so
much of a headache.  We used 2 SSG550 devices as a proof of concept and the one thing which annoyed me to no end was the complete and
total crap options within then VPN configuration.  When I typically set up a VPN, I use a SonicWall NSA or E-class device (yes I know
hiss boo) or an ASA.  Saying that the Juniper was lacking was a complete understatement.  I personally would completely avoid even
attempting VPN failover within a Juniper device.  I will say they are rock solid though for generic firewall functionality, just try
to keep the config simple or they turn into giant slow dogs.

>> - In fact I think I'm asking what are my options with regard to failover between one Internet connection and the other?

Considering you have 4x 100mbit lines, have you looked at BGP?  Even if you drop line 2 and its associated backup, you have 2x 100mbit
lines.  Or even if you have 3 unique carriers with a 100mbit from each of them it makes BGP very appealing.  I think this would be an
ideal situation for a BGP setup using a couple of small routers.  You could probably get away with something as small as a Cisco 3825
for each connection (purely redundancy).  If the Cisco name scares you Juniper routers are great as well.  Don’t forget Vyatta!

If you do BGP, you have 1 VPN to configure, you have 1 tunnel to configure, there is no VPN failover configuration and hopefully you
are not pushing more than 1 subnet across the VPN otherwise you end up doing a route based VPN instead of a policy based VPN and you
will be significantly happier.  That’s a Juniper headache for another day however.

>> I'm hoping to figure out whether adding an extra Internet connection actually gives us that much, in fact whether it justifies the
>> complexity and spend.

What you really need to know to ask this is how much is Mr. Customer going to yell and scream if someone cuts only internet connection
and it's going to be down for about 3 days while they patch the conduit and fiber.

>> Many Thanks for your comments.
>> Adel

More information about the NANOG mailing list