carping about CARP

Fri Nov 30 05:52:11 UTC 2012

I can't seem to recall anyone griping about this here on our august
little list but google finds that I'm by no means the first to have
been burned by an unholy interaction between VRRP and CARP.

Let's skip the protocol discussions (same protocol number and uses
multicast) [*] and go straight to the behavioral observations.

I turned on VRRP this evening on a pair of routers.  All of a sudden a
CARP instance between a pair of pfSense boxes in the rack (which I
didn't even know was there) invited itself to the party and started
flailing all over the place and causing oscillating packet loss for
anything that was going off-segment.

Note that the Ciscos didn't exhibit any untoward behavior, and there
were "passwords" on the VRRP sessions too.  Meanwhile, the pfSense box
spazzed out and filled its dmesg logs with stuff like:

arp: 192.0.2.1 moved from 00:00:0c:xx:xx:01 to 00:00:5e:xx:xx:01 on em1
arp: 192.0.2.1 moved from 00:00:5e:xx:xx:01 to 00:00:0c:xx:xx:01 on em1

(no other hosts on the segment were logging such activity)

Looks like CARP is a bit loose about believing stuff coming in over
the wire.  Seems a bit out of character for OpenBSD, but maybe these
days it's considered all good so long as such a malfunction only
causes an outage, not a core dump.

Anyway, word to the wise, CARP and VRRP is a bit of a dangerous mixture.

-r

[*] The OpenBSD side of the story can be read at
http://en.wikipedia.org/wiki/Common_Address_Redundancy_Protocol#No_official_Internet_protocol_number

Seems that there is a lesson to be learned here:

"o hai, we wrote this software but can not be bothered to follow your
process or formally write up the protocol, plz to be giving us a
protocol number" ain't gonna fly.