Strange Cisco 6503 problem

Dean Belev dbelev at gmail.com
Thu Jan 28 12:56:19 UTC 2010


Hi all,

I experienced a strange problem with one of our Cisco 6503 routers - 
right after the terminal PC connected to the router via console is 
rebooted the router reboots itself.
Even when there is no Eth connection to the PC the situations is  the 
same - reboot follows.

I tried to check the messages the router sends:

*: %CPU_MONITOR-6-NOT_HEARD: CPU_MONITOR messages have not been heard 
for 31 seconds [1/0]
:: No EOBC input, dump debuginfo, interval 10, times 3
: %CPU_MONITOR-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 
60 seconds [1/0]
: %CPU_MONITOR-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 
90 seconds [1/0]
: %FIB-3-FIBDISABLE: Fatal error, slot/cpu 1/0: IPC Failure: timeout
: %CPU_MONITOR-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 
120 seconds [1/0]
: %CPU_MONITOR-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 
150 seconds [1/0]*

All the possible reasons according to Cisco that may caused that case 
will be checked soon because the router is in operational mode although 
some of them looks strange:

/*CPU_MONITOR-3-TIMED_OUT or CPU_MONITOR-6-NOT_HEARD
Problem

The switch reports these error messages:
     CPU_MONITOR-3-TIMED_OUT: CPU monitor messages have failed, 
resetting system
     CPU_MONITOR-6-NOT_HEARD: CPU monitor messages have not been heard 
for [dec] seconds
Description
These messages indicate that CPU monitor messages have not been heard 
for a significant amount of time. A time-out most probably occurs,
which resets the system.
[dec] is the number of seconds.
The problem possibly occurs because of these reasons:
     *      Badly seated line card or module
     *      Bad ASIC or bad backplane
     *      Software bugs
     *      Parity error
     *      High traffic in the Ethernet out of band channel (EOBC) channel
       The EOBC channel is a half duplex channel that services many 
other functions, which includes Simple Network Management Protocol (SNMP)
     traffic and packets that are destined to the switch. If the EOBC 
channel is full of messages because of a storm of SNMP traffic,
     then the channel is subjected to collisions. When this happens, 
EOBC is possibly not able to carry IPC messages.
     This makes the switch display the error message.

     Workaround

Reseat the line card or module. If a maintenance window can be 
scheduled, reset the switch in order to clear any transient issues.
#########################################
Error Message
%ERR_DET-5-ERR_DET_NO_EOBC_INPUT: No EOBC input, dump debuginfo, 
interval %u, times %u
Explanation    No Ethernet Out-of-Band Channel (EOBC) input was 
received. Debugging information will be dumped.
Recommended Action    No action is required.*/

I'm curious if some of you faced such a problem - reboot of the router 
caused by the console connection.

Here is some brief info about the router:

*Supervisor:*
Mod Ports Card Type                                               Model
--- ----- -------------------------------------- ------------------ 
-----------
   1    9  Supervisor Engine 32 8GE (Active)      WS-SUP32-GE-3B

*Processor:*
cisco WS-C6503-E (R7000) processor

*IOS:*
IOS (tm) s3223_rp Software (s3223_rp-ADVENTERPRISEK9_WAN-M), Version 
12.2(18)SXF8, RELEASE SOFTWARE (fc2)

Thank you in advance for all your replays.

Best~



More information about the NANOG mailing list