Netflow on SUP720-3BXL

Andy Bierlair globichen at gmail.com
Sat Mar 14 20:55:51 CDT 2009


I’m trying to run netflow on one of our Cisco core routers (SUP720-3BXL),
but I think I am hitting some limitations because of this:



%EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
Utilization [99%]



The setup of netflow looks like this:



  ip flow-cache entries 524288

  mls aging fast time 5 threshold 32

  mls aging long 300

  mls aging normal 60

  mls netflow usage notify 80 300

  mls flow ip full

  no mls flow ipv6

  mls nde sender version 5

  no mls verify ip checksum

  no mls acl tcam share-global



  ip flow-export source Loopback0

  ip flow-export version 5 origin-as

  ip flow-export destination <ip> <port>



Then I have this enabled on all border interfaces/vlans (peering / transit /
other core routers) that are of interest for my stats:



  ip route-cache flow



Some more details about the problem:



#sh mls netflow table-contention detailed Earl in Module 5 Detailed Netflow
CAM (TCAM and ICAM) Utilization
================================================

TCAM Utilization             :   100%

ICAM Utilization             :   13%

Netflow TCAM count           :   262033

Netflow ICAM count           :   17

Netflow Creation Failures    :   4822220

Netflow CAM aliases          :   1





#sh mls netflow table-contention aggregate Earl in Module 5 Aggregate
Netflow CAM Contention Information
=============================================

Netflow Creation Failures    :   130003616

Netflow Hash Aliases         :   4





I understand that the TCAM is full, but what can I do against it? This is a
busy core router:



Aggregated traffic: 7-8 GBIT/s

Packets per Second: 1.0 - 1.2 Million



I wouldn't mind analyzing only every 10th or 100th flow, which seems to be a
common practice.



Any good piece of advice is welcome.



Thanks!



-
Andy



More information about the NANOG mailing list