GNi/365main Above.net peering problems?
rusty at hodge.com
Sat Aug 30 15:55:28 CDT 2008
Received this update for GNi:
> At this time we believe we have found a Cisco day 0 network
> We have 20+ routers in our core network - 6 of the 20 have the
> identical route
> processor and IOS version. These 6 have been affected in 3 separate
> geographical locations in the past several days. The network issues
> from simple SNMP failure to loss of BGP or OSPF communications.
> These are
> creating black holes intermittently across our network. We are
> experiencing as
> much as 20% of networks not available at this time.
> We have been coordinating with Cisco for the past two days and have
> patches to address the problem. Overall, we have determined that
> the fastest,
> surest path to restoring 100% network normality is to replace these
> 6 routers.
> We will continue this process to replace these 6 routers.
> Two of the six affected routers have already been replaced and
> clients have
> been moved. We are now replacing the remaining 4 affected routers.
> We know that this week has been very painful and frustrating for
> you. We will
> provide a detailed and open RFO Reason For Outage post mortem
> document as soon
> as we have the replaced the affected routers.
> Also, I will be updating all customers who have opened tickets at
> least every
> two hours from this time to ensure that all customers have the
> latest status
> information. If you would prefer a phone call status report, please
> note it
> on the ticket and I will also give you a call to answer any additional
> questions you may have. In addition, you may call the NOC at your
> at (415) 979-9786.
> Thank you for your patience.
More information about the NANOG