Alternative to NetFlow for Measuring Traffic flows

Tue Dec 17 06:53:26 UTC 2002

Quantifiable Proof and "Peering Profiles"...see below.

At 08:53 PM 12/16/2002 -0800, Joe Wood wrote:

>On Mon, 16 Dec 2002, Joe Abley wrote:
>
> > I think the idea was to say "well, from the mrtg graph, the difference
> > between this circuit with all my _9327_ traffic and this circuit
> > without any _9327_ traffic, at what I might reasonably estimate their
> > peak time to be, looks to be about 2 megs or so".
> >
> > It's a pretty crude measure, but it does have the advantage of
> > requiring no more than mrtg and a route-map to set up.

Right, it is crude, but in an economy where business decisions require 
"Quantifiable *Proof*", this is quantifiable and easy to do. Some Peering 
Coordinators are  putting together business plans now for peering at the IX 
that includes the #'s of Mbps of peering traffic, and e-mail confirmation 
from the peers at the IX that they will indeed peer with them at the IX. 
Smart customers; if they exceed the breakeven point then peering makes 
sense. A lot more work up front than it used to be.

>It is also useful as a supplement to netflow statistics, as sort of a
>verification to your flow data. Sometimes due to design, operating
>conditions, etc netflow data is not always the most reliable and/or
>meaningful.
>
>As an example:
>
>You run two main types of border router platforms. On one platform you
>must sample netflow @ 1% due to performance limitations. On the other
>platform there is no sampling functionality built into the software.
>This creates an immediate skew of data, unless software is created to
>sample the flows coming off the second platform.
>
>Now take into account that your traffic is mainly outbound from your
>network, which means that you need to ignore vendor best practice
>and enable flow caching on your core (internal) facing interfaces to
>measure the traffic flowing out of your network.
>
>So, in order for you to get any kind of traffic statistics for a peer,
>you've got to spend many hours distilling data manually, doing AS
>aggregations, and create a possibly unstable networking environment.
>
>No big deal, right?
>
>It may be crude, but sometimes it can be the most reliable _available_
>method to tell how much traffic is going to the ISP and ISP customers.

Joe is absolutely right here, and this still represents a common scenario 
and problem for the peering community.

Another approach I have been thinking about is to generate "Peering 
Profiles" for the community...here is how it works. Let's say I work with a 
few Internet Gaming companies and find that the netflow stats show a 
certain pattern, or profile of traffic destinations. Maybe I find that
2% to Cox
3% to Shaw
2% to Comcast
5% to Roadrunner
2% to Adelphia
and the next top 20 ASes represent the next 10% of traffic.

Anonymized, this "Peering Profile" for Internet Gaming companies can 
probably be applied to other Internet Gaming companies and can provide a 
rough idea of good targets for peering and how much traffic can be expected 
at a peering point, as a percentage of their total traffic. Empirically, 
these top traffic destinations and volumes have been large enough, 10's of 
Mbps each, generally more than enough to justify peering a an IX where the 
breakeven point is 10-30Mbps. The design of the tool/template is pretty 
obvious from there.

Side Note: See all the trouble we go through because traffic flow 
measurement is still non-trivial? If the netflow data is available at 
ingress/egress points, I was pointed to http://ehnt.sourceforge.net/ as a 
good freeware tool for evaluating and translating the netflow raw data.

Bill