Techniques for passive traffic capturing
ross at kallisti.us
Tue Jun 24 14:44:33 UTC 2008
On Tue, Jun 24, 2008 at 01:19:03PM +1200, Nathan Ward wrote:
> I see little point in aggregating tapped traffic, unless you have only
> a small amount of it and you're doing it to save cost on monitoring
> network interfaces - but is that saved cost still a saving when you
> factor in the cost of the extra 3750s in the middle? I'd guess no.
Thanks for all the info Nathan - lots of good leads in your email.
Let me include some more information.
The problem is finding a way to multiplex that traffic from the
optical tap to multiple things that want to peek at it. The
remote-span trick solves that, as well as integrating media
converters. 3750 is nice since you can stack em up and mix/match the
SFP and copper ports.
For example - we have an FCP box from Internap. It wants to see
mirrored traffic so it can watch for TCP setup problems and try to
find blackholes. It takes 10G feeds of aggregated transit links.
Then, we want to do some passive IDS analysis. But snort can only
really only handle 600-800Mbps before it starts saturating CPU
(not multithreaded...) - so one collector per gigE transit seems
We'd like to generate flow data out of our forwarding plane since
we use 6500s to pull in border transit links. The Netflow on those
boxes is terrible. pmacct does a much better job, but it needs to see
all the traffic out of band.
> Note that for a single GE link, you'd need 2GE of remote span backhaul
> (one GE in each direction).
We're mostly a content network, very few eyeballs. Our ingress
traffic is negligable compared to egress, which makes the problem
> Matrix switches aren't useful for your case, as you're talking about
> monitoring for trending etc. I think. Matrix switches are good when
> you have lots of links, and want to be able to switch between them. Is
> the cost of matrix switch ports worth the saving in GE interfaces on
I guess what made me look at them is their ability to multiplex the
stream of data. Take it from an optical tap, spit the same data out
of multiple ports.
The remote-span trick seems to do the same thing, so I'm wondering
where the gotcha is. If there's an advantage to using something like
the Matrix switches, I'd love to know that now.
> The above is based on the assumption you're using PCs for monitoring,
> the economics of aggregating tap traffic may make more sense if you're
> using some fancy monitoring platform.
Yea - the fact that we have both makes the aggregation method look
good. The FCP takes 10G aggregated feeds. The PCs will want single
gig views of the transit links.
> If you find that you need lots of GE interfaces per PC or something,
> and are saturating the PCI bus, look at DAG cards from Endace. They're
> designed for passive monitoring, and will send you only headers and do
> BPF in hardware. I looked at these for a similar project, but didn't
> bother as it was cheaper to buy more PC chassis' and commodity GE
> cards. They can do 10GE monitoring, so if you need several 10GE's per
> chassis I'd recommend these.
Ah the Endace gear looks really interesting. Thanks for the pointer!
ross at kallisti.us
"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
More information about the NANOG