Alternative to NetFlow for Measuring Traffic flows
William B. Norton
wbn at equinix.com
Tue Dec 17 06:53:26 UTC 2002
Quantifiable Proof and "Peering Profiles"...see below.
At 08:53 PM 12/16/2002 -0800, Joe Wood wrote:
>On Mon, 16 Dec 2002, Joe Abley wrote:
> > I think the idea was to say "well, from the mrtg graph, the difference
> > between this circuit with all my _9327_ traffic and this circuit
> > without any _9327_ traffic, at what I might reasonably estimate their
> > peak time to be, looks to be about 2 megs or so".
> > It's a pretty crude measure, but it does have the advantage of
> > requiring no more than mrtg and a route-map to set up.
Right, it is crude, but in an economy where business decisions require
"Quantifiable *Proof*", this is quantifiable and easy to do. Some Peering
Coordinators are putting together business plans now for peering at the IX
that includes the #'s of Mbps of peering traffic, and e-mail confirmation
from the peers at the IX that they will indeed peer with them at the IX.
Smart customers; if they exceed the breakeven point then peering makes
sense. A lot more work up front than it used to be.
>It is also useful as a supplement to netflow statistics, as sort of a
>verification to your flow data. Sometimes due to design, operating
>conditions, etc netflow data is not always the most reliable and/or
>As an example:
>You run two main types of border router platforms. On one platform you
>must sample netflow @ 1% due to performance limitations. On the other
>platform there is no sampling functionality built into the software.
>This creates an immediate skew of data, unless software is created to
>sample the flows coming off the second platform.
>Now take into account that your traffic is mainly outbound from your
>network, which means that you need to ignore vendor best practice
>and enable flow caching on your core (internal) facing interfaces to
>measure the traffic flowing out of your network.
>So, in order for you to get any kind of traffic statistics for a peer,
>you've got to spend many hours distilling data manually, doing AS
>aggregations, and create a possibly unstable networking environment.
>No big deal, right?
>It may be crude, but sometimes it can be the most reliable _available_
>method to tell how much traffic is going to the ISP and ISP customers.
Joe is absolutely right here, and this still represents a common scenario
and problem for the peering community.
Another approach I have been thinking about is to generate "Peering
Profiles" for the community...here is how it works. Let's say I work with a
few Internet Gaming companies and find that the netflow stats show a
certain pattern, or profile of traffic destinations. Maybe I find that
2% to Cox
3% to Shaw
2% to Comcast
5% to Roadrunner
2% to Adelphia
and the next top 20 ASes represent the next 10% of traffic.
Anonymized, this "Peering Profile" for Internet Gaming companies can
probably be applied to other Internet Gaming companies and can provide a
rough idea of good targets for peering and how much traffic can be expected
at a peering point, as a percentage of their total traffic. Empirically,
these top traffic destinations and volumes have been large enough, 10's of
Mbps each, generally more than enough to justify peering a an IX where the
breakeven point is 10-30Mbps. The design of the tool/template is pretty
obvious from there.
Side Note: See all the trouble we go through because traffic flow
measurement is still non-trivial? If the netflow data is available at
ingress/egress points, I was pointed to http://ehnt.sourceforge.net/ as a
good freeware tool for evaluating and translating the netflow raw data.
More information about the NANOG