Netflow on SUP720-3BXL

Sun Mar 15 01:55:51 UTC 2009

I’m trying to run netflow on one of our Cisco core routers (SUP720-3BXL),
but I think I am hitting some limitations because of this:

%EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
Utilization [99%]

The setup of netflow looks like this:

  ip flow-cache entries 524288

  mls aging fast time 5 threshold 32

  mls aging long 300

  mls aging normal 60

  mls netflow usage notify 80 300

  mls flow ip full

  no mls flow ipv6

  mls nde sender version 5

  no mls verify ip checksum

  no mls acl tcam share-global

  ip flow-export source Loopback0

  ip flow-export version 5 origin-as

  ip flow-export destination <ip> <port>

Then I have this enabled on all border interfaces/vlans (peering / transit /
other core routers) that are of interest for my stats:

  ip route-cache flow

Some more details about the problem:

#sh mls netflow table-contention detailed Earl in Module 5 Detailed Netflow
CAM (TCAM and ICAM) Utilization
================================================

TCAM Utilization             :   100%

ICAM Utilization             :   13%

Netflow TCAM count           :   262033

Netflow ICAM count           :   17

Netflow Creation Failures    :   4822220

Netflow CAM aliases          :   1

#sh mls netflow table-contention aggregate Earl in Module 5 Aggregate
Netflow CAM Contention Information
=============================================

Netflow Creation Failures    :   130003616

Netflow Hash Aliases         :   4

I understand that the TCAM is full, but what can I do against it? This is a
busy core router:

Aggregated traffic: 7-8 GBIT/s

Packets per Second: 1.0 - 1.2 Million

I wouldn't mind analyzing only every 10th or 100th flow, which seems to be a
common practice.

Any good piece of advice is welcome.

Thanks!

-
Andy