Linux Router distro's with dual stack capability

Fri Feb 12 00:41:29 UTC 2010

On Thu, Feb 11, 2010 at 03:46:13PM -0800, Kevin Oberman wrote:
> Polling is excellent for low speed lines, but for Gig and faster, most
> newer interfaces support interrupt coalescing. This easily resolves the
> issue in hardware as interrupts are only issued when needed but limited
> to a reasonable rate, Polling does not use interrupts, but consumes
> system resources regardless of traffic.
> 
> FreeBSD has supported polling for a long time (V6?) and interrupt
> coalescing since some release of V7. (Latest release is V8.)

I'm pretty sure it's been around for a lot longer than that. I seem to
recall playing with both back in 4.x. Of course interrupt coalescing is
mostly a function of the NIC (though some driver involvement is required
to take advantage of it), so the quality of the implementations have
varied significantly over the years. The first generation GE NICs which
offered it didn't do a particularly good job with it though, so for
example it was still possible to cripple a box with high interrupt
rates while the same box would be perfectly fine with polling.

That said, I think your use case for polling is backwards. As you say, 
"normally" the NIC fires off an interrupt every time a packet is 
received, and the kernel stops what it is doing to process the new 
packet. On a low speed (or at least low traffic) interface this isn't a 
problem, but as the packet/sec rate increases the amount of time wasted 
as interrupt processing "overhead" becomes significant. For example, 
even a GE interface is capable of doing 1.488 million packets/sec.

By switching to a polling based model, you switch off the interrupt 
generation completely and simply check the NIC for new packets a set 
rate (for example, 1000 times/sec). This gives you a predictable and 
consistent CPU use, so even if you had 1.488M/s interrupts coming in you 
would still only be checking 1000 times/sec. If you did less than 
1000pps it would be a net increase in CPU, but if you do more (or ever 
risk doing more, such as during a DoS attack) it could be a net benefit. 
This is makes the most sense for people doing a lot of traffic 
regardless.

Of course the downside is higher latency, since you're delaying the 
processing of the packet by some amount of time after it comes in. In 
the 1000 times/sec example above, you could be delaying processing of 
your packet by up to 1ms. For most applications this isn't enough to 
cause any harm, but it's something to keep in mind. Interrupt coalescing 
works around the problem of large interrupt rates by simply having the 
NIC limit the number of interrupts it generates under load, giving you 
the benefits of low-latency processing and low-interrupt rate under high 
load. I haven't played with this stuff in many many years, so I'm sure 
modern interrupt coalescing is much better than it used to be, and the 
extra work of configuring polling and dealing with the potential 
latency/jitter implications isn't worth the benefits for most people. :)

-- 
Richard A Steenbergen <ras at e-gerbil.net>       http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)