BGP Design question.

Thu Jun 23 03:42:30 UTC 2011

I second the static routes, specially from a simplicity standpoint.  Add
in a pair of layer two switches to simplify further:

     +--------+    +--------+
     | Peer A |    | Peer A |  <-Many carriers. Using 1 carrier
     +---+----+    +----+---+    for this scenario.
         |eBGP          | eBGP
         |              |
     +---+----+iBGP+----+---+
     | Router +    + Router |  <- Routers. Not directly connected
     +-+------+    +------+-+
       |                  |
     +-+------+    +------+-+
     |L2Switch|----|L2Switch|  <- Layer 2 switches, can be stacked
     +--------+    +--------+
       |                  |
     +-+------+    +------+-+
     |Act. FW |----|Pas. FW |  <-Firewalls Active/Passive.
     +--------+    +--------+

You can lose all of the left leg, or all of the right leg, and still be
up.  If you want to complicate things, you can add crossing links
between it all, but again, beyond BGP and VRRP, this is a very simple
design you can easily troubleshoot at 3AM.  It's also much easier to
document the troubleshooting steps (so you can go on vacation and
someone else can solve without calling you) and test upgrades.

You can nearly evenly split the traffic by having a VRRP VIP on each
edge router, with the other router backing up the first.  The firewalls
can have two static routes, one to each VIP, and this will roughly
load-balance the traffic out on a packet basis.  As you peer with the
same ISP, this will work just fine.  If they have an outage, your edge
routers will learn, and even if the circuit drops it'll know, and
basically the VIP will just redirect traffic to the other router.

Now all your firewalls have to do is maintain stateful session
information, not OSPF.

If you had two different ISPs (especially if they are not roughly evenly
connected), then not having intelligence of the BGP paths in your
firewalls can cause an extra hop when it hits router with the longer
path, which will redirect it to the router with the shorter path.

Speaking from a Cisco/HSRP point of view, you could be more intelligent
(re:more complicated, and complication means harder troubleshooting and
more documentation needed) during problem periods by having the VIP move
routers automatically based on the WAN link dropping and/or a route
beyond it being lost (others can comment to if VRRP supports this).
This would save one hop to the "broken" router when the BGP path or WAN
is down.

Jason Roysdon

On 06/22/2011 06:07 PM, Bret Palsson wrote:
> On Wed, Jun 22, 2011 at 5:33 PM, PC <paul4004 at gmail.com> wrote:
> 
>> Who makes the firewall?
>>
>>
> Juniper SSG. We use NSRP and replicate all the RTOs. We have hitless on the
> Firewalls, have for years. We're now peering with our own carriers vs. using
> our datacenter's mix.
> 
> A static route from the junipers to the VIP (VRRP) is probably the way to
> go. I think.
> 
> To make this work and be "hitless", your firewall vendor must support
>> stateful replication of routing protocol data (including OSPF).  For
>> example, Cisco didn't support this in their ASA product until version 8.4 of
>> code.
>>
>> Otherwise, a failover requires OSPF to re-converge -- and quite frankly,
>> will likely cause some state of confusion on the upstream OSPF peers, loss
>> of adjacency, and a loss of routing until this occurs.  It's like someone
>> just swapped a router with the same IP  to the upstream device -- assuming
>> your active/standby vendor's implementation only presents itself as one
>> device.
>>
>> However, once this is succesful your current failover topology should work
>> fine -- even if it takes some time to failover.
>>
>> In my opinion though, unless the firewall is serving as "transit" to
>> downstream routers or other layer 3 elements, and you need to run OSPF to it
>> (And through it) as a result, it's often just easier to static default route
>> out from the firewall(s) and redistribute a static route on the upstream
>> routers for the subnets behind the firewalls.  It also helps ensure
>> symmetrical traffic flows, which is important for stateful firewalls and can
>> become moderatly confusing when your firewalls start having many interfaces.
>>
>>
>>
>>
>> On Wed, Jun 22, 2011 at 4:27 PM, Bret Palsson <bret at getjive.com> wrote:
>>
>>> Here is my current setup in ASCII art. (Please view in a fixed width
>>> font.) Below the art I'll write out the setup.
>>>
>>>
>>>     +--------+    +--------+
>>>     | Peer A |    | Peer A |  <-Many carriers. Using 1 carrier
>>>     +---+----+    +----+---+    for this scenario.
>>>         |eBGP          | eBGP
>>>         |              |
>>>     +---+----+iBGP+----+---+
>>>     | Router +----+ Router |  <-Netiron CERs Routers.
>>>     +-+------+    +------+-+
>>>       |A   `.P    A.'    |P   <-A/P indicates Active/Passive
>>>       |      `.  .'      |      link.
>>>       |        ::        |
>>>     +-+------+'  `+------+-+
>>>     |Act. FW |    |Pas. FW |  <-Firewalls Active/Passive.
>>>     +--------+    +--------+
>>>
>>>
>>> To keep this scenario simple, I'm multihoming to one carrier.
>>> I have two Netiron CERs. Each have a eBGP connection to the same peer.
>>> The CERs have an iBGP connection to each other.
>>> That works all fine and dandy. Feel free to comment, however if you think
>>> there is a better way to do this.
>>>
>>> Here comes the tricky part. I have two firewalls in an Active/Passive
>>> setup. When one fails the other is configured exactly the same
>>> and picks up where the other left off. (Yes, all the sessions etc. are
>>> actively mirrored between the devices)
>>>
>>> I am using OSPFv2 between the CERs and the Firewalls. Failover works just
>>> fine, however when I fail an OSPF link that has the active default route,
>>> ingress traffic still routes fine and dandy, but egress traffic doesn't.
>>> Both Netiron's OSPF are setup to advertise they are the default route.
>>>
>>> What I'm wondering is, if OSPF is the right solution for this. How do
>>> others solve this problem?
>>>
>>>
>>> Thanks,
>>>
>>> Bret
>>>
>>>
>>> Note: Since lately ipv6 has been a hot topic, I'll state that after we get
>>> the BGP all figured out and working properly, ipv6 is our next project. :)
>>>
>>>
>>>
>>
> 