scaling linux-based router hardware recommendations

Mike Hammett nanog at ics-il.net
Tue Jan 27 02:20:07 UTC 2015


Different (configuration) strokes for different folks. I look at a Cisco interface now and say, "Who the hell would use this?" despite my decade old Cisco training. 

I was corrected offlist that Vyatta does do MPLS now... but I can't find anything on it doing VPLS, so I guess that's still out. 

The 5600's license (according to their SDNCentral performance report) appears to be near $7k whereas MT you can get a license for $80. 




----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

----- Original Message -----

From: "Paul S." <contact at winterei.se> 
To: nanog at nanog.org 
Sent: Monday, January 26, 2015 8:10:54 PM 
Subject: Re: scaling linux-based router hardware recommendations 

Like Mike mentioned, the feature list in RouterOS is nothing short of 
impressive -- problem is that pretty much everything in there is 
inherently buggy. 

That and one hell of a painful syntax-schema to work with too. 

On 1/27/2015 午前 10:57, Tony Wicks wrote: 
> And the solution to this issue is - http://routerboard.com/ or http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don't scoff until you have tried it, the price/performance is pretty staggering if you are in the sub 20gig space. 
> 
> -----Original Message----- 
> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Mike Hammett 
> Sent: Tuesday, 27 January 2015 2:44 p.m. 
> To: nanog at nanog.org 
> Subject: Re: scaling linux-based router hardware recommendations 
> 
> Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the "cloud-scale" data centers deploying this stuff still have more traditional big iron at their cores. 
> 
> The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 
> 
> 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. 
> 
> I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. 
> 
> Haven't gotten a quote from AlcaLu yet. 
> 
> Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. 
> 
> The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. 
> 
> Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been "looking into" DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. 
> 
> 
> 
> 
> ----- 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> ----- Original Message ----- 
> 
> From: "Mehmet Akcin" <mehmet at akcin.net> 
> To: "micah anderson" <micah at riseup.net> 
> Cc: nanog at nanog.org 
> Sent: Monday, January 26, 2015 6:06:53 PM 
> Subject: Re: scaling linux-based router hardware recommendations 
> 
> Cumulus Networks has some stuff, 
> 
> http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf 
> 
> Pretty decent presentation with more details you like. 
> 
> Mehmet 
> 
>> On Jan 26, 2015, at 8:53 PM, micah anderson <micah at riseup.net> wrote: 
>> 
>> 
>> Hi, 
>> 
>> I know that specially programmed ASICs on dedicated hardware like 
>> Cisco, Juniper, etc. are going to always outperform a general purpose 
>> server running gnu/linux, *bsd... but I find the idea of trying to use 
>> proprietary, NSA-backdoored devices difficult to accept, especially 
>> when I don't have the budget for it. 
>> 
>> I've noticed that even with a relatively modern system (supermicro 
>> with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server 
>> adapters, and 16gig of ram, you still tend to get high percentage of 
>> time working on softirqs on all the CPUs when pps reaches somewhere 
>> around 60-70k, and the traffic approaching 600-900mbit/sec (during a 
>> DDoS, such hardware cannot typically cope). 
>> 
>> It seems like finding hardware more optimized for very high packet per 
>> second counts would be a good thing to do. I just have no idea what is 
>> out there that could meet these goals. I'm unsure if faster CPUs, or 
>> more CPUs is really the problem, or networking cards, or just plain 
>> old fashioned tuning. 
>> 
>> Any ideas or suggestions would be welcome! 
>> micah 
>> 





More information about the NANOG mailing list