scaling linux-based router hardware recommendations

Mon Jan 26 23:44:37 UTC 2015

Has anyone tested these setups with something more beefy like dual Xeons of Sandybridge or later vintage? Waiting to hear back from one NIC vendor (HotLava) what they think can be done on larger hardware setups. Put in two big Xeons and you're looking at 24 cores to work with as opposed to the <8 on the desktop versions. The newer ones would also have PCIe 3, which would overcome bus speed limitations in PCIe 2. 

Realistic to put 6x - 12x 10GigEs into a server with that much beef and expect it to perform well? What vintage of core ix do you run, Faisal? 

----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

----- Original Message -----

From: "Faisal Imtiaz" <faisal at snappytelecom.net> 
To: "micah anderson" <micah at riseup.net> 
Cc: nanog at nanog.org 
Sent: Monday, January 26, 2015 5:27:55 PM 
Subject: Re: scaling linux-based router hardware recommendations 

Hi Micah, 

There is a segment in the Hardware Side of the industry that produces "Network Appliances". 
(Folks such as Axiomtek, Lanner Electronics, Caswell Networks, Portwell etc etc) 

These appliances are commonly used as a commercial (OEM) platform for a variety of uses.. 
Routers, Firewalls, Specialized network applications etc. 

Our internal testing ( informal), matches up with the commonly quoted PPS handling by the different product vendors who incorporate these appliances in their network product offerings. 

i3/i5/i7 (x86) based network appliances will forward traffic as long as pps does not exceed 1.4million 
(In our testing we found the pps to be limiting factor and not the amount of traffic being moved) 
(will easily handle 6G to 10G of traffic 

Core2duo (x86) based network appliances will forward traffic as long as pps does not exceed 600,0000 pps 
(will easily handle 1.5G to 2G of traffic) 

Atom based (x86) network appliances will forward traffic as long as pps does not exceed 250,000 pps. 

---------------------------------------- 

Of course, if you start to bog down the router with lots of NAT/ACL/ Bridge Rules (i.e. the CPU has to get involved in traffic management) then your actual performance will be degraded. 

Regards. 

Faisal Imtiaz 
Snappy Internet & Telecom 
7266 SW 48 Street 
Miami, FL 33155 
Tel: 305 663 5518 x 232 

Help-desk: (305)663-5518 Option 2 or Email: Support at Snappytelecom.net 

----- Original Message ----- 
> From: "micah anderson" <micah at riseup.net> 
> To: nanog at nanog.org 
> Sent: Monday, January 26, 2015 5:53:54 PM 
> Subject: scaling linux-based router hardware recommendations 
> 
> 
> Hi, 
> 
> I know that specially programmed ASICs on dedicated hardware like Cisco, 
> Juniper, etc. are going to always outperform a general purpose server 
> running gnu/linux, *bsd... but I find the idea of trying to use 
> proprietary, NSA-backdoored devices difficult to accept, especially when 
> I don't have the budget for it. 
> 
> I've noticed that even with a relatively modern system (supermicro with 
> a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server 
> adapters, and 16gig of ram, you still tend to get high percentage of 
> time working on softirqs on all the CPUs when pps reaches somewhere 
> around 60-70k, and the traffic approaching 600-900mbit/sec (during a 
> DDoS, such hardware cannot typically cope). 
> 
> It seems like finding hardware more optimized for very high packet per 
> second counts would be a good thing to do. I just have no idea what is 
> out there that could meet these goals. I'm unsure if faster CPUs, or 
> more CPUs is really the problem, or networking cards, or just plain old 
> fashioned tuning. 
> 
> Any ideas or suggestions would be welcome! 
> micah 
> 
> 