few big monolithic PEs vs many small PEs

Thu Jun 27 23:23:58 UTC 2019

I've ran into many providers where they had routers in the top 10 or 15 markets... and that was it. If you wanted a connection in South Bend or Indianapolis or New Orleans or Ohio or... you were backhauled potentially hundreds of miles to a nearby big market. 

More smaller POPs reduces the tromboning. 

More smaller POPs means that one POP's outage isn't as disastrous on the traffic rerouting around it. 

----- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

----- Original Message -----

From: adamv0025 at netconsultings.com 
To: nanog at nanog.org 
Sent: Wednesday, June 19, 2019 3:22:45 PM 
Subject: few big monolithic PEs vs many small PEs 

Hi folks, 

Recently I ran into a peculiar situation where we had to cap couple of PE 
even though merely a half of the rather big chassis was populated with 
cards, reason being that the central RE/RP was not able to cope with the 
combined number of routes/vrfs/bgp sessions/etc.. 

So this made me think about the best strategy in building out SP-Edge 
nowadays (yes I'm aware of the centralize/decentralize pendulum swinging 
every couple of years). 
The conclusion I came to was that *currently the best approach would be to 
use several medium to small(fixed) PEs to replace a big monolithic chasses 
based system. 
So what I was thinking is, 
Yes it will cost a bit more (router is more expensive than a LC) 
Will end up with more prefixes in IGP, more BGP sessions etc.. -don't care. 
But the benefits are less eggs in one basket, simplified and hence faster 
testing in case of specialized PEs and obviously better RP CPU/MEM to port 
ratio. 
Am I missing anything please? 

*currently, 
Yes some old chassis systems or even multi-chassis systems used to support 
additional RPs and offloading some of the processes (e.g. BGP onto those) 
-problem is these are custom hacks and still a single OS which needs 
rebooting LC/ASICs when being upgraded -so the problem of too many eggs in 
one basket still exists (yes cisco NCS6k and recent ASR9k lightspeed LCs are 
an exception) 
And yes there is the "node-slicing" approach from Juniper where one can 
offload CP onto multiple x86 servers and assign LCs to each server (virtual 
node) - which would solve my chassis full problem -but honestly how many of 
you are running such setup? Exactly. And that's why I'd be hesitant to 
deploy this solution in production just yet. I don't know of any other 
vendor solution like this one, but who knows maybe in 5 years this is going 
to be the new standard. Anyways I need a solution/strategy for the next 3-5 
years. 

Would like to hear what are your thoughts on this conundrum. 

adam 

netconsultings.com 
::carrier-class solutions for the telecommunications industry:: 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190627/36c2611f/attachment.html>