Time Warner outage?

Thu Aug 28 13:42:01 UTC 2014

Sounds more likely than "we broke the dhcp server".

Sent from my T-Mobile 4G LTE Device

-------- Original message --------
From: Chris Garrett <chris at aperturefiber.com>
Date: 08/28/2014 6:39 AM (GMT-08:00)
To: "Naslund, Steve" <SNaslund at medline.com>
Cc: nanog at nanog.org
Subject: Re: Time Warner outage?

I have never seen BGP drop peers because of a DHCP failure.

I would love to see the breakdown on that.

TW NOC had said it was a Level 3 fiber maintenance gone bad that took down their peering connections when we called in at the time of the event.

Can anyone confirm one way or the other?

On Aug 28, 2014, at 8:29 AM, Naslund, Steve <SNaslund at medline.com> wrote:

> Something sounds really unlikely about that.  Lack of DHCP would not cause reachability problems except for the clients.  The trace below looks like a transit connection that should be unaffected by DHCP.  Looks more like a routing issue.  Also sounds unlikely that one DHCP server would be covering this large a scope.
>
> Steven Naslund
> Chicago IL
>
> -----Original Message-----
> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Chris Lane
> Sent: Thursday, August 28, 2014 8:23 AM
> To: Stephen Satchell
> Cc: nanog at nanog.org
> Subject: Re: Time Warner outage?
>
> I heard the following,
>
> It was actually an engineer doing maintenance on the dhpc server that stopped customers from getting an IP address when the connected to the network between 5:30 and 7:00.  The funny part is upper Managment heard about it on the today show
>
>
> On Thu, Aug 28, 2014 at 8:46 AM, Stephen Satchell <list at satchell.net> wrote:
>
>> This just keeps getting better and better:
>>
>> "Yahoo Logo
>> Will be right back...
>>
>> Thank you for your patience.
>>
>> Our engineers are working quickly to resolve the issue."
>>
>> My upstream is Charter Business...
>>
>> On 08/28/2014 04:05 AM, Chris Garrett wrote:
>>> I believe this is what is commonly referred to in our industry as a
>> “resume generating event”.
>>>
>>>
>>> On Aug 27, 2014, at 8:24 AM, Alexandru Suciu
>>> <alex.suciu.sum at gmail.com>
>> wrote:
>>>
>>>> Can't wait to see the postmortem report on this one.
>>>>
>>>> On 8/27/2014 3:05 PM, Adam Greene wrote:
>>>>> Came back for us, too, at about 7am.
>>>>>
>>>>> Outage was a biggie this time:
>>>>>
>>>>>
>> https://news.yahoo.com/time-warner-cable-suffering-massive-nationwide-
>> outage
>>>>> -110549606.html
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of David
>> Coulson
>>>>> Sent: Wednesday, August 27, 2014 6:59 AM
>>>>> To: Rick Coloccia
>>>>> Cc: <nanog at nanog.org>
>>>>> Subject: Re: Time Warner outage?
>>>>>
>>>>> Just came back up for me.
>>>>>
>>>>> Sent from my iPad
>>>>>
>>>>>> On Aug 27, 2014, at 6:48 AM, Rick Coloccia <coloccia at geneseo.edu>
>> wrote:
>>>>>>
>>>>>> BGPMON shows my routes falling off the net at around 5:49am.
>>>>>>
>>>>>> We now sit at their mercy....
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On Aug 27, 2014, at 6:46 AM, "Adam Greene"
>>>>>>> <maillist at webjogger.net>
>>>>> wrote:
>>>>>>> Same here. Seems like no traffic is exiting TWC:
>>>>>>>
>>>>>>> Tracing route to ns03.savvis.net [204.70.25.234] over a maximum
>>>>>>> of
>> 30
>>>>>>> hops:
>>>>>>>
>>>>>>> 1     1 ms     1 ms     1 ms  10049.webjogger.net [204.8.80.49]
>>>>>>> 2    27 ms     6 ms     5 ms  cpe-24-29-112-25.nyc.res.rr.com
>>>>>>> [24.29.112.25]
>>>>>>> 3    12 ms     8 ms     7 ms  rdc-69-193-225-74.nyc.bc.twcable.com
>>>>>>> [69.193.225.74]
>>>>>>> 4     9 ms     9 ms    12 ms  rdc-69-193-225-137.nyc.bc.twcable.com
>>>>>>> [69.193.225.137]
>>>>>>> 5     9 ms    10 ms     8 ms  rdc-69-193-225-222.nyc.bc.twcable.com
>>>>>>> [69.193.225.222]
>>>>>>> 6     *        *        *     Request timed out.
>>>>>>> 7     *        *        *     Request timed out.
>>>>>>> 8     *        *        *     Request timed out.
>>>>>>> 9     *        *        *     Request timed out.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Rick
>>>>>>> Coloccia
>>>>>>> Sent: Wednesday, August 27, 2014 6:43 AM
>>>>>>> To: nanog at nanog.org
>>>>>>> Subject: Re: Time Warner outage?
>>>>>>>
>>>>>>> My whole campus (~10000 users) is down... Since roughly 6am.
>>>>>>> TWC is our upstream.
>>>>>>>
>>>>>>> --
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>>> On Aug 27, 2014, at 6:28 AM, Rob Barbeau
>>>>>>>> <rob.barbeau at gmail.com>
>> wrote:
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> I have a branch office in Syracuse,NY that appears to be down
>>>>>>>> at the moment that uses a time warner business connection for
>>>>>>>> internet
>> access.
>>>>>>>>
>>>>>>>> -rob
>>>>>>>> On Aug 27, 2014 5:20 AM, "David Hubbard"
>>>>>>>> <dhubbard at dino.hostasaurus.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey all, anyone else having issues with Time Warner
>>>>>>>>> residential or business connections?  One of our offices is
>>>>>>>>> down and the route is not currently in bgp.
>>>>>>>>> http://downdetector.com/status/time-warner-cable
>>>>>>>>> shows thousands of reports of outages on the consumer side
>>>>>>>>> starting an hour or so ago so I figure it's a larger issue
>>>>>>>>> than just my one office; couldn't reach anyone by phone.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> David
>>>>
>>>
>>
>>
>
>
> --
> //CL