NANOG36-NOTES 2006.02.14 talk 2 Netflow Visualization Tools

Roland Dobbins rdobbins at
Wed Feb 15 06:39:13 UTC 2006

Roland Dobbins - that's me asking about the time intervals for the  
bins and the TCP flags stuff.


Note that 5-minute bins may not always be optimal for opsec - 5  
minutes minimum to see something happening and then 5 minutes to see  
if your mitigation action was effective is a long time.  With NetFlow- 
based anomaly-detection systems, the active flow timeout value is  
generally turned down to one minute; the operator may -choose- to  
suppress certain types of alarms for a set period, or configure  
threshold-transition delays, but being stuck at a practical minimum  
of 10 minutes between detection and confirmation of mitigation due to  
data-conversion overhead (the collected flow telemetry must be  
converted into another format prior to analysis) may be an issue, in  
some circumstances.

On Feb 14, 2006, at 8:13 PM, Vicky Røde wrote:

> Hash: SHA1
> thanks for taking notes.
> comments in-line:
> Matthew Petach wrote:
>> 2006.02.14 talk 2 Netflow tools
>> Bill Yurcik
>> byurcik at
>> NVisionIP and VisFlowConnect-IP
>> probably a dozen tools out there, this is just
>> two of them.  Concenses is there's something to
>> this.
>> They're an edge network, comes into ISP domain,
>> their tools are used by entities with many
>> subnet blocks.
>> Overview
>> Project Motifivation
>> Netflows for Security
>> Two visualization tools
>>  NVisionIP
>>  VisFlowConnect-IP
>> Summary
>> Internet Security:
>> N-Dimensional Work Space
>> large--already lots of data to process
>> complex--combinatorics explode quickly
>> time dynamics--things can change quickly!
>> Visualizations can help!
>>  in near-realtime
>>  overview-browse-details on demand
>> People are wired to do near-realtime processing
>> of visual information, so that's a good way to
>> present information for humans.
>> HCI says use overview-browse-details paradigm.
>> Netflows for security
>> can identify connection-oriented stats to see
>> things like attacks, DoS, DDoS, etc.
>> Most people don't use the data portion of the
>> flow field, the first 64 bytes, they just look
>> at header info or aggregated flow records.
>> Can spot how many users are on your system at
>> a given time, to schedule upgrades.
>> Who are your top talkers?
>> How long do my users surf?  What are people using
>> the network for?
>> Where do users go?   Where did they come from?
>> Are users following the security policy?
>> What are the top N destination ports?
>> Is there traffic to vulnerable hosts?
>> Can you identify and block scanners/bad guys?
>> This doesn't replace other systems like syslog, etc.;
>> it integrates and works alongside them.
>> architecture slide for NCSA.
>> Can't really do sampled view for security, so probably
>> need distributed flow collector farm to get all the
>> raw data safely.
>> Two visualization tools:
>> NVisionIP, VisFlowConnect-IP
>> focus on quick overview of tools
>> 3 level hierarchical tool;
>> galaxy view (small multiple view) ((machine view))
>> Galaxy is overview of the whole network.
>> color and shape of dots is each host in a network.
>> settable parameters for each dot.
>> Animated toolbar and clock show changes over time
>> in the galaxy.
>> Lets you get high-level content quickly and easily.
>> Domain view lets you drill in a bit more; small
>> multiple view looks at the traffic within the
>> block.
>> upper histogram is lower, well known ports; lower
>> histogram is ports over 1024
>> You can click on a given multiple view entry to
>> delve into one machine.
>> Many graphs for each machine in the most detailed
>> view.
>> well known ports first, then rest of ports (sorted)
>> then source and destination traffic broken out.
>> Designed for class Bs.
>> VisFlowConnectDownload.html
>> 3 vertical lines, comes from edge network perspective;
>> middle line is edge network to manage.  You set range
>> of networks you care about.  Outside lines are people
>> sourcing or sinking traffic to you, from outside
>> domains.
>> There's a time axis, traffic only shown for the slice
>> of time currently under consideration.
>> Uses VCR-like controls to move time forward/backward
>> Lets you see traffic/interactivity, drill into that
>> domain, see host level connectivity flows.
>> Shows MS Blaster virus traffic as an example.
>> Example 2, a scan example.  Just because it looks
>> like one IP hitting many others doesn't mean it's
>> really a security incident, though; could be a
>> cluster getting traffic.
>> web crawlers hitting NCSA web servers make for
>> a very charateristic pattern over time.
>> Summary
>> Netflows analysis is non-trivial,
>> NVisionIP
>> VisFlowConnect-IP
>> lots of references listed in very fine blue font.
>> Avi Freedman, Akamai, Argus was mentioned a lot; it
>> lets you grab symmetric netflows, but also does TCP
>> analysis, shows some performance data as well.  not
>> sure if people are studying the impact of correlating
>> argus data with flow data.
>> Roland Douta? of Cisco; many people are using netflow
>> to track security issues.  They now have ingress and
>> egress flow data on many of their platforms.
>> In reading paper describing it, there's data conversion
>> that needs to happen into an internal format that
>> nVision can understand.  It reads log files at the
>> moment, takes about 5 minutes to process files.  Lets
>> them take different file data sources, make the tool
>> for visualization independent of the input format.
>> They can read large files, but there is a performance
>> hit when doing it.
>> Are they planning on doing further work on the tool
>> to collect TCP flags, for frags, drop traffic, etc?
>> They've looked at it, but they leave it to IDS tools
>> for flag activity.  Might be of interest to consider
>> for future versions of the tools.
>> Last question came up, echoed about argus.
>> Question about interactivity, they are working on
>> feedback through tools.  Question about alarming
>> on patterns; but once you start alarming or putting
>> up visual indicators, it distracts from rest of
>> the overall pattern, you tend to miss other information.
> - ----------------
> the last part was me, virendra rode from riverdomain. my question was
> mostly related to a possibility of setting priority bit(s) in order to
> control (rate-limit, if you will) session(s) that could lead to  
> congestion.
> since argus is already integrated and performs traffic auditing (i
> think) setting priority bit(s) would be a nice feature to integrate  
> down
> the path. then again, i understand this is a performance   
> monitoring tool.
> that's all.
> regards,
> /virendra
> Version: GnuPG v1.2.5 (MingW32)
> Comment: Using GnuPG with Thunderbird -
> iD8DBQFD8qpUpbZvCIJx1bcRAnzaAKCsI29SetdMSJaLr3LR01MGp87CmACgnCEf
> 7RDnyaGsad++GevXjt2MIQY=
> =/55T

Roland Dobbins <rdobbins at> // 408.527.6376 voice

      Everything has been said.  But nobody listens.

                    -- Roger Shattuck

More information about the NANOG mailing list