BGP Enabled transit in Chicago (River North) and equipment recommendation

Tue Sep 3 19:54:07 UTC 2019

Ubiquiti's EdgeRouter Lite is equipped with 512 MiB of DDR2 memory, of which after startup, roughly 491 MiB can be utilized. 119 MiB of the remaining memory are allocated by the base of the router already, which leaves you with a remainder of 372 MiB memory. Memory usage depends on the architecture for objects, for example there's a large difference between x86 and x86_64, since on x86_64, the compiler will generally use 64bit boundaries to be faster; the ERL runs on a MIPS64 architecture, which will have a similar trade-off. To get to the point, let's have a quick look at the components using memory: bgpd, zebra, kernel. Roughly 180 MiB of memory are required to keep a single full table in bgpd alone, leaving you with 192 MiB of free memory. Accounting further, zebra will eat at least another 100 MiB for exporting the BGP RIB to the Kernel (FIB), leaving you with 100 MiB. At this point, you have a mere 92 MiB left for fitting the routes into the kernel, and to leave room for RX buff
ers on sockets.

I don't see full tables happening from a memory perspective on the EdgeRouter Lite, you would want to look at something with at least 2 GiB of memory to keep the whole system running smoothly, and when using Quagga and Zebra, that's still aimed rather low. FRRouting at this point uses 2 GiB for 4 full tables on an x86 system, without any magic attached.
Having kept it unmentioned, the EdgeRouter Lite has a dual-core with 500 MHz, and surely your BGP updates processing isn't offloaded, hence you will pretty quickly kill the whole router when you flood it with a full table, unless you set very low queue sizes, which isn't really reliable though since you generally want BGP to converge fast - not after a period of 15 minutes with the CPU sitting on 100%.
You might want to install something like OpenWRT (which I don't know the possibility of on an ERL), and run BIRD if you're tied to a low memory footprint, however, in a base vendor-generic setup of the ERL, it's beyond my understanding why one would even suggest running a full table on it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190903/fab29c9b/attachment.html>