layer 3 switch debate

Fri Sep 27 09:28:39 UTC 2002

On Fri, 27 Sep 2002, Stephen J. Wilcox wrote:

> > When designing an all IP network requiring mostly Ethernet interfaces, the
> > logical conclusion is to specify layer 3 switches (instead of routers). The
> > cost per port and functionality requirements make a layer 3 switch the
> > perfect choice.

> First questions would be how large is this network in terms of interfaces and
> traffic flows and how is it distributed. If its small use cheap L2 switches, if
> its on one or a couple of sites with not many hosts why do you need L3 in there
> at all, stick to L2..

Agree with the not many hosts part. You especially don't want customer
hosts to connect directly to your layer 2 core because it gets very messy.
But the main problem with layer 2 is that it handles redundant links so
badly: spanning tree simply disables them.

> If its larger and more distributed then you need to aggregate up anyway so I'd
> imagine its cheaper to use plain L3 routers connecting the L2 LAN across
> intersite WANs

> Needs more quantifying to find any conclusion but I dont see that an "all IP
> network" requires a L3 switch network!

Presumably, you'll need at least one router (I mean a device configured to
do layer 3 forwarding, whatever it may be called) to connect to your
transit(s).

> > However, the rule of thumb in the IP community is that routers are superior
> > to layer 3 switches and should be utilized instead, especially when
> > considering core type functionality.

> I think we have a terminolgy issue here.. assuming a L3 switch is a device which
> uses routing decisions to influence a switching process then you get this on
> current Cisco routers.. the L3 makes the routing decision on the first packet in
> the stream but then additional frames are switched.

This is traditional "layer 4 (or multilayer) switching" or flow-based
routing/switching.  Be very careful with this because it can blow up in
your face if there are very many new flows every second, which is
typically the case for any serious level of WWW traffic.

> But that aside I think you
> mean what vendors call "L3 switches or L4 switches" which are like a L2 switch
> but go into higher layer protocols to influence the switching decision and
> perform other features most commonly load balancing.

The terms are used so loosely these days that you really need to
investigate and not infer functionality from the name alone.

> > Does this rule of thumb still apply considering the modern layer 3 switches
> > available? If not, why? What makes a layer 3 switch sub-standard to a pure
> > router? Any quantitative analysis you could provide would be greatly
> > appreciated.

> Most commonly seems to be interoperability, the switches do their own job fine
> in their own isolated environment but they cant act as a "ISP router".. in my
> experience then tend to have odd bugs and behave slightly unexpectedly when say
> for example routing OSPF or BGP. Altho this is probably a chicken and egg - if
> more people tried to use them perhaps the vendors would fix the code!

I've been running OSPF on a pretty old Extreme switch for years now in a
small network (hand full of routers, several hundred routes) and no
problems at all. I've had mixed reports on BGP on Extreme and Riverstone,
but on Foundry it seems to work well for at least several people I've
talked with. All of these boxes will route IP very fast and relatively
cheap. If money is really tight, you could consider PC's running your
favorite Unix flavor and Zebra. The functionality is fine, but the
hard- and software fails more often so you need to put in more redundancy.

Another good way to go would be just a couple of large Cisco or Juniper
boxes for all the layer 3 stuff (limiting broadcast domains, filtering and
routing protocols) and connect everything else using switches and VLANs.

> > I realize your answer may depend on device position within the network. I am
> > comparing a router to a layer 3 switch as a core routing device, an EBGP
> > border router and access device. Remember, my network is comprised of mostly
> > Ethernet interfaces (FW, GE) and the occasional DS1 and DS3 interface.

Core routers typically don't do any filtering and the BGP setup (if any)
is straightforward, so switch-like routers are good here. For access you
need filtering, which many layer 2 switches won't do. Multilayer stuff is
also good here, since it gives you many ports and good performance. If you
don't want to have these boxes talk BGP to your customers you can simply
backhaul BGP customer subnets over a VLAN to one or more "real" routers
elsewhere. I wouldn't necessarily recommend routers with a switching
heritage as border routers since this tends to stress the BGP
implementations the most.

In any case, you'll be taking the road less travelled so test your stuff
real good before deployment.