High throughput bgp links using gentoo + stipped kernel

Sun May 19 20:37:06 UTC 2013

(oops, I keep forgetting to send with my nanog identity..)

On 19. mai 2013 17:48, Nick Khamis wrote:
> We do use a statefull iptables on our router, some forward rules...
> This is known to be on of our issues, not sure if having a separate
> iptables box would be the best and only solution for this?

Ah, statefullness/conntrack .. once you load it you kinda lost already.. 
Sorry. Any gains from other tunables will likely be dwarfed by the cpu 
cycles spent by the kernel to track all connections. The more diverse 
the traffic the more it will hurt. Connection tracking is just 
inherently non-scalable (and fragile - by the way.)

However, the cheapest and simplest is probably just to throw more modern 
hardware at it. A Xeon E3 (or two for redudancy ;)) is quite cheap..

The long term, scalable solution is a deeper network like you hinted at, 
with statefullness - if really needed at all - pushed as close to your 
edge and as far away from your border as possible. But.. More boxes, 
more to manage, more power, more stuff that can fail, more redudancies 
needed.. adds up.

Then again if you are close to gig actual traffic already, you might 
want to at least think about future scalability..

<snip>
> Any ideas of the setup??? Maybe as far as naming some chipset, interface?
> And xserver that is the best candidate. Will google.. :)

The big shift to integrated (and fast) I/O happened around 2008 IIRC, 
anything introduced after that is usually quite efficient at moving 
packets around, at least if Intel based. Even desktop i3/i5/i7 platforms 
can do 10gig as long as you make sure you put the network chips/cards on 
the cpu pcie controllers lanes. With anything new its hard to go wrong.

xserver?? xserve? That is quite old..

>> Curious about vmstat output during saturation, and kernel version too.
>> IPv4 routing changed significantly recently and IPv6 routing performance
>> also improved somewhat.
>
> Will get that output during peak on monday for you guys. Newest kernel
> 3.6 or 7...

Good. That is at least fairly recent and has most of the more modern 
networking stuff (and better defaults)