few big monolithic PEs vs many small PEs

Fri Jun 21 07:36:59 UTC 2019

Hey,

> From: Tarko Tikan
> Sent: Thursday, June 20, 2019 8:28 AM
> 
> hey,
> 
> > For availability I think it is best approach to do many small edge
> > devices.
> 
> This is also great for planned maintenance. ISSU has not really worked out for
> any of the vendors and with two small devices you can upgrade them
> independently.
>
Yup I guess no one is really using ISSU in production, and even with ISSU, currently, most of the NPUs on the market need to be power-cycled to load a new version of microcode so there's packet loss on data-plane anyways. 
 
> Great for aggregation, enables you to dual-home access devices into two
> separate PEs that will never be down at the same time be it failure or
> planned maintenance (excluding the physical issues like power/cooling but
> dual-homing to two separate sites is always problematic for eyeball
> networks).
> 
Actually this is an interesting point you just raised.
(note: The assumption for the below is single-homed customers, as the dual-homed customer would probably what to be at least site diverse and pay premium for that service)
So what is the primary goal of us using the aggregation/access layer? It's to achieve better utilization of the expensive router ports right? (hence called aggregation)
And indeed there are cases where we connect customers directly on to the PEs, but then it's somehow ok for a line-card to be part of just a single chassis (or a PE).
Now let's take a step even further what if the line-card is not inside the chassis anymore -cause it's a fabric-extender or a satellite card.
Why all of a sudden we'd be uncomfortable again to have it part of just a single chassis (and there are tons of satellite/extender topologies to prove that this is a real concern among operators).
So to circle back to a standalone aggregation device -should we try and complicate the design by creating this "fabric" (PEs "spine" and aggregation devices "leaf") in an attempt to increase resiliency or shall we treat each aggregation device as unitary indivisible part of a single PE as if it was a card in a chassis -cause if the economics worked It would be a card in a chassis?

adam