Cisco IOS MPLS VPN Bug

Sat Mar 12 07:31:54 UTC 2011

Hello All,

A customer of our's has had two major outages due to a bug in Cisco's
software.  The network is an MPLS VPN environment.  The bug has occurred on
our 6500(s) with VS-S720-10G Supervisors running
s72033-adventerprisek9_wan-mz.122-33.SXH6.bin.   These routers
are configured as BGP route-reflectors.  The first time the bug
hit occurred when a new route-reflector client was added.  The second time
was due to a topology change on the network.  On both occasions the only fix
was removing the peer configurations on the RR and re-applying.  Niether
soft nor hard clears on the BGP neighbors worked, only the config removal.
 Once re-applied life was good.

The bug itself was with the BGP updates sent by the RR.  During the outage
these updates did not include the Route Target Extended Community required
by the route-reflector clients which identifies which VRF the route belongs
too.  Below is output on a client during the outage and after the config
yanking.  Notice the mysterious disappearance of the RT community.

Looking to see if anyone has seen this issue particularly with this version
of code.  TAC is trying to tell me that this was a bug in a previous version
but is fixed in the code I am running.  Huh?  Been running around in circles
with them for a month so this is my act of desperation.   Also if anyone is
running a similar environment without issue I would be very interested in
what version of code your using.

Thanks to all who took the time to read this email... Happy Friday.

*BAD:*

CLN-MWB-2811-01#sh ip bgp vpnv4 all 10.180.33.22

BGP routing table entry for 102:102:10.180.33.20/30, version 1763889

Paths: (2 available, best #2, no table)

  Advertised to update-groups:

        1

  Local

    10.180.20.1 (metric 9) from 10.180.20.3 (10.180.20.3)

      Origin incomplete, metric 0, localpref 100, valid, internal

      Extended Community: RT:102:102

      Originator: 10.180.20.1, Cluster list: 0.0.255.245

      mpls labels in/out nolabel/16

  Local

    10.180.20.1 (metric 9) from *10.180.20.1* (10.180.20.1)

      Origin incomplete, metric 0, localpref 100, valid, internal, best

      mpls labels in/out nolabel/16

*GOOD:*

CLN-MWB-2811-01#sh ip bgp vpnv4 all 10.180.33.22

BGP routing table entry for 102:102:10.180.33.20/30, version 1765931

Paths: (2 available, best #1, table AMS)

  Advertised to update-groups:

        1

  Local

    10.180.20.1 (metric 9) from *10.180.20.1* (10.180.20.1)

      Origin incomplete, metric 0, localpref 100, valid, internal, best

      *Extended Community: RT:102:102*

      mpls labels in/out nolabel/16

  Local

    10.180.20.1 (metric 9) from 10.180.20.3 (10.180.20.3)

      Origin incomplete, metric 0, localpref 100, valid, internal

      Extended Community: RT:102:102

      Originator: 10.180.20.1, Cluster list: 0.0.255.245

      mpls labels in/out nolabel/16

Cheers,

-- 
Joe Renwick
IP Network Consultant, CCIE #16465
GO NETFORWARD!
Direct: 619-800-2055, Emergency Support: 800-719-0504
Is your network moving you forward?