Level(3) ex-twtelecom midwest packet loss (4323)

James Sink jsink at phone.com
Fri Aug 28 16:57:02 UTC 2015


Seemed reasonably accurate to me.


[image: Phone.com] <http://www.phone.com/>James SinkSenior Network Engineer
jsink at phone.com(800) 997-9179 x506Try Phone.com free!
<http://www.phone.com/?_tracking_id=10132>CONFIDENTIALITY NOTICE: This
e-mail and any attachments are for the exclusive and confidential use of
the intended recipient. If you received this in error, please do not read,
distribute, or take action in reliance upon this message. Instead, please
notify us immediately by return e-mail and promptly delete this message and
its attachments from your computer system.

On Fri, Aug 28, 2015 at 8:45 AM, Mel Beckman <mel at beckman.org> wrote:

> Blake,
>
> There's no call to be blatantly offensive like that.
>
> -mel
>
> > On Aug 28, 2015, at 8:35 AM, Blake Dunlap <ikiris at gmail.com> wrote:
> >
> > I'll just leave this here....
> >
> >
> https://honestnetworker.wordpress.com/2013/11/04/the-true-meaning-behind-most-rfos/
> >
> >> On Fri, Aug 28, 2015 at 8:26 AM, Jason Canady <jason at unlimitednet.us>
> wrote:
> >> Mike, I would take it to mean someone screwed something up and they
> don't
> >> want to admit to it.  :-)  That's just a guess.
> >>
> >> --
> >>
> >> Jason Canady
> >> Unlimited Net, LLC
> >> Responsive, Reliable, Secure
> >>
> >> www.unlimitednet.us
> >> jason at unlimitednet.us
> >> twitter: @unlimitednet
> >>
> >>
> >>> On 8/28/15 12:00 AM, Mike Hammett wrote:
> >>>
> >>> 08/28/2015 3:08 AM GMT
> >>> Event Conclusion Summary
> >>>
> >>> Start: August 27, 2015 13:20 GMT
> >>> Stop: August 28, 2015 00:00 GMT
> >>>
> >>> Root Cause: A protocol issue impacted IP services in multiple markets.
> >>> Fix Action: Adjustments were made to clear the errors.
> >>>
> >>> Summary:
> >>> The IP NOC began investigating the root cause with Tier III Technical
> >>> Support. It was reported that the issue was causing packet loss for
> >>> customers. Operations Engineering teams were engaged, and Field
> Services
> >>> were dispatched to a site in Chicago, IL to assist with investigations.
> >>> Troubleshooting identified a protocol issue, and Operations Engineering
> >>> worked with Tier III Technical Support to perform adjustments on the
> links.
> >>> It was confirmed that the errors cleared. The traffic load was also
> lowered
> >>> on cards in Chicago to alleviate any further issues. Should any
> additional
> >>> impact be experienced, please contact the Level 3 Technical Service
> Center.
> >>>
> >>> What the hell is a "protocol issue"?
> >>>
> >>> I'm not an idiot, you can tell me specifically what happened...
> >>>
> >>>
> >>>
> >>>
> >>> -----
> >>> Mike Hammett
> >>> Intelligent Computing Solutions
> >>> http://www.ics-il.com
> >>>
> >>> ----- Original Message -----
> >>>
> >>> From: "Ryan Gelobter" <ryan.g at atwgpc.net>
> >>> To: "Mel Beckman" <mel at beckman.org>
> >>> Cc: "<nanog at nanog.org>" <nanog at nanog.org>
> >>> Sent: Thursday, August 27, 2015 3:14:59 PM
> >>> Subject: Re: Level(3) ex-twtelecom midwest packet loss (4323)
> >>>
> >>> If you have access to the Level3 portal you should see ticket #9639047
> >>> under Network Events now.
> >>>
> >>> Event Summary:IP Network Event ~ Multiple Markets
> >>>
> >>> 08/27/2015 8:05 PM GMT
> >>>
> >>> Level 3 Tier III and Operations Engineering teams have identified
> Internet
> >>> Protocols dropping, affecting customer services. Restoration efforts
> are
> >>> in progress, but an estimated time of restoral is not available at this
> >>> time.
> >>>
> >>> 08/27/2015 6:36 PM GMT
> >>>
> >>> IP and Transport Tier III, Operations Engineering and Field Services
> >>> continue collaboratively working the issue.
> >>>
> >>>
> >>> 08/27/2015 4:59 PM GMT
> >>>
> >>> Operations Engineering is engaged and Field Services is on site in
> >>> Chicago,
> >>> IL investigating the issue.
> >>>
> >>>
> >>> 08/27/2015 4:38 PM GMT
> >>>
> >>> The engineers are currently migrating traffic in efforts of restoring
> >>> services while troubleshooting continues. Field Services is being
> >>> dispatched to a Chicago, IL site to assist.
> >>>
> >>> 08/27/2015 4:21 PM GMT
> >>>
> >>> IP services are affected across multiple markets and the root cause is
> >>> currently under investigation. The IP NOC and IP and Transport Tier III
> >>> are
> >>> actively troubleshooting and working to isolate the cause. The
> engineers
> >>> have detected peering issues which are resulting in packet loss for
> >>> customers. Please be advised that updates will be provided at minimum
> of
> >>> hourly unless otherwise noted.
> >>
>



More information about the NANOG mailing list