Peering VLANs and MAC addresses

Thu Nov 10 00:01:43 UTC 2005

On Wed, 9 Nov 2005, Robert Kiessling wrote:

> Which rule would you suggest for the IXP? The naive "connect 
> only routers" wouldn't do of course in nowaday's world of 
> hybrids.

I've been following this with interest:

* How do you differentiate between a switch/router and a router? A lot of 
people are deploying C76xx as peering routers these days because of the 
cheapness of the ports (including 10G) on that particular problem.

* A lot of people with vendor J kit take a dot1q encaps GigE to a switch 
and break out 100M ports, because 100M ports aren't very cost effective on 
the M-series platform.

The rule on LINX always used to be "no switches", until the switch/router 
convergence dilemma.

There was also an issue of keeping business (especially business we wanted
to keep!) on the IX, that wanted to connect through LAN extensions and
various other more colourful methods, so we had to find a way of 
permitting this while mitigating the risks.

There's a fairly competitive IX environment in London, and the other
exchanges didn't seem to be too bothered by switches, especially if it
helped them attract participants from the larger exchange ;).

So, the rule at LINX is now "one port, one source MAC, any more and we 
shut you down".

This deals with all the usual evils, such as ad-hoc extensions (or resold
connections) to the peering platform (a prime concern), incoming
STP/CDP/VTP, etc., and broken routers/switches (which end up spewing crap
- C75xx used to melt spectacularly in this way). At the same time it
permits switch/routers, and various LAN/Ethernet extension services, as
long as they are sensibly managed and configured.

We used to police this policy semi-manually, but now the switch vendors do 
decent hardware-based port-security/mac-locking functionality, so that 
does it for us, and actually does it pretty well.

- The switch learns the first address received on the interface, which 
should be the first ingress frame (usually an ARP generated by the router 
sending a BGP Open), and remembers it (with a 3 minute ageing time).

- This has the affect of applying an acl to the port (in hardware), which 
permits traffic from the "good" address, and drops frames from other 
addresses. 

- Should more than 100 different source MACs be learned (99 of which will 
be filtered and dropped) on the interface, the port will then log a 
critical violation and shut the port down.

It works pretty well, it prevents all the usual badness we'd normally 
associate with switches on the IXP.

So at the end of the day, it looks like we've been able to find a happy
medium, maintaining decent "hygiene", while being able to let people
indulge in deploying switches if they so choose.

Cheers,
Mike