few big monolithic PEs vs many small PEs

Thu Jun 27 19:41:53 UTC 2019

On 27 June 2019 16:26:03 BST, Mark Tinka <mark.tinka at seacom.mu> wrote:
>
>
>On 27/Jun/19 10:58, James Bensley wrote:
>
>> Hi Adam,
>>
>> Over the years I have been bitten multiple times by having fewer big
>> routers with either far too many services/customers connected to them
>> or too much traffic going through them. These days I always go for
>> more smaller/more routers than fewer/larger routers.
>>
>> One experience I have made is that when there is an outage on a large
>> PE, even when it still has spare capacity, is that the business
>impact
>> can be too much to handle (the support desk is overwhelmed, customers
>> become irate if you can't quickly tell them what all the impacted
>> services are, when service will be restored, the NMS has so many
>> alarms it’s not clear what the problem is or where it's coming from
>> etc.).
>>
>> I’ve seen networks place change freeze on devices, with the exception
>> of changes that migrate customers or services off of the PE, because
>> any outage would create too great an impact to the business, or risk
>> the customers terminating their contract. I’ve also seen changes
>> freeze be placed upon large PEs because the complexity was too great,
>> trying to work out the impact of a change on one of the original PEs
>> from when the network was first built, which is somehow linked to
>> virtually every service on the network in some obscure and
>> unforeseeable way.
>
>I would tend to agree when the edge routers are massive, e.g., boxes
>like the Cisco ASR9922 or the Juniper MX2020 are simply too large, and
>present a real risk re: that level of customer aggregation (even for
>low-revenue services such as Broadband). I don't think I'd ever justify
>buying these towers to aggregate customers, mainly due to the risk.
>
>For us, even the MX960 is too big, which is why we focus on the MX480
>(ASR9906 being the equivalent). It's a happy medium between the small
>and large end of the spectrum.
>
>And as I mentioned before, we just look at a totally different box for
>100Gbps customers.
>
>Mark.

Yeah, if you want to name specific boxes then yes I've made similar experiences with the same boxen. Even the MX960 is slightly too big for a PE depending on how you load it (port combinations).

Large boxes like the MX2020, ASR9922, NCS6K, etc. these can only reasonably be used as P nodes in my opinion.

Cheers,
James.