NANOG36-NOTES 2006.02.14 talk 2 Netflow Visualization Tools
rdobbins at cisco.com
Wed Feb 15 06:39:13 UTC 2006
Roland Dobbins - that's me asking about the time intervals for the
bins and the TCP flags stuff.
Note that 5-minute bins may not always be optimal for opsec - 5
minutes minimum to see something happening and then 5 minutes to see
if your mitigation action was effective is a long time. With NetFlow-
based anomaly-detection systems, the active flow timeout value is
generally turned down to one minute; the operator may -choose- to
suppress certain types of alarms for a set period, or configure
threshold-transition delays, but being stuck at a practical minimum
of 10 minutes between detection and confirmation of mitigation due to
data-conversion overhead (the collected flow telemetry must be
converted into another format prior to analysis) may be an issue, in
On Feb 14, 2006, at 8:13 PM, Vicky Røde wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> thanks for taking notes.
> comments in-line:
> Matthew Petach wrote:
>> 2006.02.14 talk 2 Netflow tools
>> Bill Yurcik
>> byurcik at ncsa.uiuc.edu
>> NVisionIP and VisFlowConnect-IP
>> probably a dozen tools out there, this is just
>> two of them. Concenses is there's something to
>> They're an edge network, comes into ISP domain,
>> their tools are used by entities with many
>> subnet blocks.
>> Project Motifivation
>> Netflows for Security
>> Two visualization tools
>> Internet Security:
>> N-Dimensional Work Space
>> large--already lots of data to process
>> complex--combinatorics explode quickly
>> time dynamics--things can change quickly!
>> Visualizations can help!
>> in near-realtime
>> overview-browse-details on demand
>> People are wired to do near-realtime processing
>> of visual information, so that's a good way to
>> present information for humans.
>> HCI says use overview-browse-details paradigm.
>> Netflows for security
>> can identify connection-oriented stats to see
>> things like attacks, DoS, DDoS, etc.
>> Most people don't use the data portion of the
>> flow field, the first 64 bytes, they just look
>> at header info or aggregated flow records.
>> Can spot how many users are on your system at
>> a given time, to schedule upgrades.
>> Who are your top talkers?
>> How long do my users surf? What are people using
>> the network for?
>> Where do users go? Where did they come from?
>> Are users following the security policy?
>> What are the top N destination ports?
>> Is there traffic to vulnerable hosts?
>> Can you identify and block scanners/bad guys?
>> This doesn't replace other systems like syslog, etc.;
>> it integrates and works alongside them.
>> architecture slide for NCSA.
>> Can't really do sampled view for security, so probably
>> need distributed flow collector farm to get all the
>> raw data safely.
>> Two visualization tools:
>> NVisionIP, VisFlowConnect-IP
>> focus on quick overview of tools
>> 3 level hierarchical tool;
>> galaxy view (small multiple view) ((machine view))
>> Galaxy is overview of the whole network.
>> color and shape of dots is each host in a network.
>> settable parameters for each dot.
>> Animated toolbar and clock show changes over time
>> in the galaxy.
>> Lets you get high-level content quickly and easily.
>> Domain view lets you drill in a bit more; small
>> multiple view looks at the traffic within the
>> upper histogram is lower, well known ports; lower
>> histogram is ports over 1024
>> You can click on a given multiple view entry to
>> delve into one machine.
>> Many graphs for each machine in the most detailed
>> well known ports first, then rest of ports (sorted)
>> then source and destination traffic broken out.
>> Designed for class Bs.
>> 3 vertical lines, comes from edge network perspective;
>> middle line is edge network to manage. You set range
>> of networks you care about. Outside lines are people
>> sourcing or sinking traffic to you, from outside
>> There's a time axis, traffic only shown for the slice
>> of time currently under consideration.
>> Uses VCR-like controls to move time forward/backward
>> Lets you see traffic/interactivity, drill into that
>> domain, see host level connectivity flows.
>> Shows MS Blaster virus traffic as an example.
>> Example 2, a scan example. Just because it looks
>> like one IP hitting many others doesn't mean it's
>> really a security incident, though; could be a
>> cluster getting traffic.
>> web crawlers hitting NCSA web servers make for
>> a very charateristic pattern over time.
>> Netflows analysis is non-trivial,
>> lots of references listed in very fine blue font.
>> Avi Freedman, Akamai, Argus was mentioned a lot; it
>> lets you grab symmetric netflows, but also does TCP
>> analysis, shows some performance data as well. not
>> sure if people are studying the impact of correlating
>> argus data with flow data.
>> Roland Douta? of Cisco; many people are using netflow
>> to track security issues. They now have ingress and
>> egress flow data on many of their platforms.
>> In reading paper describing it, there's data conversion
>> that needs to happen into an internal format that
>> nVision can understand. It reads log files at the
>> moment, takes about 5 minutes to process files. Lets
>> them take different file data sources, make the tool
>> for visualization independent of the input format.
>> They can read large files, but there is a performance
>> hit when doing it.
>> Are they planning on doing further work on the tool
>> to collect TCP flags, for frags, drop traffic, etc?
>> They've looked at it, but they leave it to IDS tools
>> for flag activity. Might be of interest to consider
>> for future versions of the tools.
>> Last question came up, echoed about argus.
>> Question about interactivity, they are working on
>> feedback through tools. Question about alarming
>> on patterns; but once you start alarming or putting
>> up visual indicators, it distracts from rest of
>> the overall pattern, you tend to miss other information.
> - ----------------
> the last part was me, virendra rode from riverdomain. my question was
> mostly related to a possibility of setting priority bit(s) in order to
> control (rate-limit, if you will) session(s) that could lead to
> since argus is already integrated and performs traffic auditing (i
> think) setting priority bit(s) would be a nice feature to integrate
> the path. then again, i understand this is a performance
> monitoring tool.
> that's all.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.5 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> -----END PGP SIGNATURE-----
Roland Dobbins <rdobbins at cisco.com> // 408.527.6376 voice
Everything has been said. But nobody listens.
-- Roger Shattuck
More information about the NANOG