Route table prefix monitoring

Fri Sep 4 16:07:04 UTC 2009

Howdy all,

I've done a bit of digging through the Google machine and the MarkMail
archive of NANOG (Which is a great resource I cannot plug enough -
http://nanog.markmail.org) and have a few vague answers, but would like
some deeper thought so I'm putting this out to the list.

We recently had an event where, unbeknownst to us, a circuit went down
and a /16 prefix inside our core routing table was withdrawn as a
consequence of adjacency disappearing with that downed circuit (the fact
that it went down without us knowing is being worked already).  This
caused a severe breakage for a legacy system that hasn't been touched in
years, and tribal knowledge couldn't explain why we were seeing that
legacy system going to a subnet that nobody knew anything about (again,
documentation is something that's being worked already as a consequence
of this).

What I'm left thinking is that it would have been great if we'd had a
snapshot of our core routing table as it stood hours or even days prior
to this event occurring, so that I could compare it with our current
"broken" state, so the team could have seen that subnet in the core
table and what the next hop was for the prefix.  Are there any tools
that people are using to track when/what prefixes are added/withdrawn
from their routing tables, or to pull the routing table as a whole at
regular intervals for storage/comparison purposes?  It looks like
there's a plugin for NAGIOS, but I'm looking for suggestions on any
other tools (commercial, open source, home grown) that we might take a
look at.  For reference, we are running Cisco as well as Juniper kit.

Feel free to drop me your thoughts off-list.

Thank you for any insight ahead of time,

-Jason "Feren" Olsen