Hi-Rise Building Fiber Suggestions

Brandon Martin lists.nanog at monmotha.net
Wed Feb 26 16:30:58 UTC 2020


On 2/25/20 10:48 PM, Abhi Devireddy wrote:
> L2 rings IMHO seem pretty brittle. I know there are L2 ring products 
> like Juniper BTI, which use ERPS and not strictly STP/RSTP to move 
> blocking ports, and those seem a little better although it's mostly 
> statically configured.

For a strict ring topology like this, I'd certainly consider E-RPS or 
similar over [R]STP if you were going to do this all at L2.  It's a lot 
more "predictable" in my experience insofar as it's harder to shoot 
yourself in the foot by mistuning some knob somewhere and getting 
behavior you don't expect.

That said, I'd be loathe to put 30+ switches on a ring even within the 
same building unless I had little other choice.

One thing that I haven't seen explored in this thread is the idea of 
doing things at L3.  If you've got 4 fibers, you can use Bi-Di optics to 
build a partial mesh/ladder/braid as you go up the building.  That can 
be a mess at L2 with (R)STP, but carving out an IGP area and doing it at 
L3 is often not nearly so ugly.  If you've got L3 switches (which are 
cheap, these days), it may be a good option, though it may also be 
annoying from an IP subnetting POV unless you overlay it with something 
like an IPv4-in-IPv6 core (MAP, 464XLAT, etc.) or an L2-in-IP overlay 
like VXLAN both of which substantially increase the conplexity of the 
situation.

Using CWDM or DWDM with 8-16 channels on either 2 or 4 trunks across 
your 4 fibers to do a more conventional home-run with multi-chassis LAG 
or similar is another reasonable option.

I'd avoid stacking 30+ switches even where you have stacking support 
over fiber especially if the switches aren't in a physical stack.  Too 
much opportunity for split-brain scenarios IMO.

I'd contribute to the "see really hard if you can just drop more fiber 
down the riser" echo chamber.
-- 
Brandon Martin



More information about the NANOG mailing list