NANOG36-NOTES 2006.02.14 talk 2 Netflow Visualization Tools

Tue Feb 14 20:00:47 UTC 2006

2006.02.14 talk 2 Netflow tools

Bill Yurcik
byurcik at ncsa.uiuc.edu

NVisionIP and VisFlowConnect-IP

probably a dozen tools out there, this is just
two of them.  Concenses is there's something to
this.

They're an edge network, comes into ISP domain,
their tools are used by entities with many
subnet blocks.

Overview
Project Motifivation
Netflows for Security
Two visualization tools
 NVisionIP
 VisFlowConnect-IP
Summary

Internet Security:
N-Dimensional Work Space

large--already lots of data to process
complex--combinatorics explode quickly
time dynamics--things can change quickly!
Visualizations can help!
 in near-realtime
 overview-browse-details on demand

People are wired to do near-realtime processing
of visual information, so that's a good way to
present information for humans.
HCI says use overview-browse-details paradigm.

Netflows for security
can identify connection-oriented stats to see
things like attacks, DoS, DDoS, etc.
Most people don't use the data portion of the
flow field, the first 64 bytes, they just look
at header info or aggregated flow records.

Can spot how many users are on your system at
a given time, to schedule upgrades.

Who are your top talkers?

How long do my users surf?  What are people using
the network for?

Where do users go?   Where did they come from?

Are users following the security policy?

What are the top N destination ports?
Is there traffic to vulnerable hosts?

Can you identify and block scanners/bad guys?

This doesn't replace other systems like syslog, etc.;
it integrates and works alongside them.

architecture slide for NCSA.

Can't really do sampled view for security, so probably
need distributed flow collector farm to get all the
raw data safely.

Two visualization tools:
NVisionIP, VisFlowConnect-IP

focus on quick overview of tools
security.ncsa.uiuc.edu/

3 level hierarchical tool;
galaxy view (small multiple view) ((machine view))

Galaxy is overview of the whole network.
color and shape of dots is each host in a network.
settable parameters for each dot.

Animated toolbar and clock show changes over time
in the galaxy.
Lets you get high-level content quickly and easily.

Domain view lets you drill in a bit more; small
multiple view looks at the traffic within the
block.
upper histogram is lower, well known ports; lower
histogram is ports over 1024

You can click on a given multiple view entry to
delve into one machine.
Many graphs for each machine in the most detailed
view.

well known ports first, then rest of ports (sorted)
then source and destination traffic broken out.

Designed for class Bs.

http://security.ncsa.uiuc.edu/distribution/VisFlowConnectDownload.html

3 vertical lines, comes from edge network perspective;
middle line is edge network to manage.  You set range
of networks you care about.  Outside lines are people
sourcing or sinking traffic to you, from outside
domains.

There's a time axis, traffic only shown for the slice
of time currently under consideration.
Uses VCR-like controls to move time forward/backward

Lets you see traffic/interactivity, drill into that
domain, see host level connectivity flows.

Shows MS Blaster virus traffic as an example.

Example 2, a scan example.  Just because it looks
like one IP hitting many others doesn't mean it's
really a security incident, though; could be a
cluster getting traffic.

web crawlers hitting NCSA web servers make for
a very charateristic pattern over time.

Summary
Netflows analysis is non-trivial,

NVisionIP
VisFlowConnect-IP

lots of references listed in very fine blue font.

http://security.ncsa.uiuc.edu/distribution/NVisionIPDownload

Avi Freedman, Akamai, Argus was mentioned a lot; it
lets you grab symmetric netflows, but also does TCP
analysis, shows some performance data as well.  not
sure if people are studying the impact of correlating
argus data with flow data.

Roland Douta? of Cisco; many people are using netflow
to track security issues.  They now have ingress and
egress flow data on many of their platforms.
In reading paper describing it, there's data conversion
that needs to happen into an internal format that
nVision can understand.  It reads log files at the
moment, takes about 5 minutes to process files.  Lets
them take different file data sources, make the tool
for visualization independent of the input format.
They can read large files, but there is a performance
hit when doing it.
Are they planning on doing further work on the tool
to collect TCP flags, for frags, drop traffic, etc?
They've looked at it, but they leave it to IDS tools
for flag activity.  Might be of interest to consider
for future versions of the tools.

Last question came up, echoed about argus.
Question about interactivity, they are working on
feedback through tools.  Question about alarming
on patterns; but once you start alarming or putting
up visual indicators, it distracts from rest of
the overall pattern, you tend to miss other information.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20060214/d4e1736b/attachment.html>