google troubles?

Wed Jul 10 18:39:24 UTC 2013

On Wed, Jul 10, 2013 at 2:18 PM, Christopher Morrow
<morrowc.lists at gmail.com> wrote:
> On Wed, Jul 10, 2013 at 12:20 PM, Mike Jackson <mike at routed.ca> wrote:
>> I just realized that it's not Google IP space (74.125.0.0/16), Rogers is
>> hijacking the DNS and resolving www.google.com to space within
>> 64.71.240.0/20 which is Rogers IP space! (note the name server set as
>> 8.8.8.8)
>>
>
> so:
>  1) rogers is hijacking traffic to 8.8.8.8
>  2) the copy of 8.8.8.8 in rogers-land is replying with incorrect
> information for google properties (at least).
>
> err... any idea if it's lying about other things too? :)
>

oops, so a polite caller noted that google often ships people at a
local 'google global cache' node if they have one... it seems like
that might be what's going on here with 8.8.8.8 sending you to 64.71
and/or 66.185 addresses.

so maybe rogers isn't doing something untoward after all!

-chris
(thanks for the headslap polite caller)

>> davinci#traceroute www.google.com
>>
>> Translating "www.google.com"...domain server (8.8.8.8) [OK]
>>
>> Type escape sequence to abort.
>> Tracing the route to www.google.com (66.185.85.29)
>>
>>   1 x.x.x.x [AS 812] 4 msec 4 msec 4 msec
>>   2 so-4-0-2.gw02.ym.phub.net.cable.rogers.com (66.185.82.129) [AS 812] 8
>> msec 8 msec 4 msec
>>   3 69.63.252.222 [AS 812] 4 msec 0 msec 8 msec
>>   4 69.63.250.162 [AS 812] 4 msec 4 msec 4 msec
>>   5  *  *  *
>>   6  *  *  *
>>   7  *  *  *
>>   8  *  *  *
>>   9  *  *  *
>>  10  *  *  *
>>
>> TOR2-CORE-R1#show ip bgp 66.185.85.29
>> BGP routing table entry for 66.185.80.0/20, version 13095115
>> Paths: (1 available, best #1, table default)
>>   Advertised to update-groups:
>>      14
>>   701 6461 812
>>     205.205.23.121 from 205.205.23.121 (137.39.8.42)
>>       Origin IGP, localpref 100, valid, external, best
>> TOR2-CORE-R1#
>>
>>
>> Thanks,
>>
>>   - Mike
>>
>>
>> On Wed, Jul 10, 2013 at 11:28 AM, Mike Jackson <mike at routed.ca> wrote:
>>
>>> I can see the Google IP space (64.71.240.0/20) from Verizon/AS701, but
>>> not from Rogers/AS812 in Toronto.  I've done a few other test traceroutes
>>> through Rogers to verify that they didn't filter UDP/ICMP.  At this point
>>> nothing would surprise me from Rogers.
>>>
>>> AS701
>>> =====
>>>
>>> TOR2-CORE-R1#traceroute www.google.com
>>> Translating "www.google.com"...domain server (8.8.8.8) [OK]
>>>
>>> Type escape sequence to abort.
>>> Tracing the route to www.google.com (74.125.26.99)
>>> VRF info: (vrf in name/id, vrf out name/id)
>>>   1 x.x.x.x [AS701] 4 msec 0 msec 0 msec
>>>   2 0.ge-11-0-0.XT4.TOR2.ALTER.NET (152.63.133.78) [AS 701] 4 msec 0 msec
>>> 4 msec
>>>   3 0.so-4-0-3.XT2.NYC4.ALTER.NET (152.63.0.73) [AS 701] 16 msec 16 msec
>>> 16 msec
>>>   4 TenGigE0-7-1-0.GW8.NYC4.ALTER.NET (152.63.21.125) [AS 701] 20 msec
>>>     TenGigE0-5-1-0.GW8.NYC4.ALTER.NET (152.63.21.73) [AS 701] 16 msec
>>>     TenGigE0-5-4-0.GW8.NYC4.ALTER.NET (152.63.18.206) [AS 701] 20 msec
>>>   5 72.14.238.232 [AS 15169] 16 msec 16 msec 20 msec
>>>   6 72.14.236.208 [AS 15169] [MPLS: Label 680976 Exp 4] 16 msec 16 msec
>>>     72.14.236.206 [AS 15169] [MPLS: Label 533197 Exp 4] 20 msec
>>>   7 209.85.249.11 [AS 15169] [MPLS: Label 16668 Exp 4] 24 msec
>>>     72.14.239.93 [AS 15169] [MPLS: Label 14644 Exp 4] 24 msec
>>>     209.85.249.11 [AS 15169] [MPLS: Label 13978 Exp 4] 24 msec
>>>   8 209.85.243.114 [AS 15169] [MPLS: Label 568789 Exp 4] 32 msec
>>>     209.85.241.222 [AS 15169] [MPLS: Label 632535 Exp 4] 32 msec 32 msec
>>>   9 216.239.48.159 [AS 15169] 36 msec 32 msec
>>>     216.239.48.59 [AS 15169] 32 msec
>>>  10 www.google.com (74.125.26.147) [AS 15169] 32 msec 32 msec 32 msec
>>>
>>>
>>>
>>> AS812 (Rogers Looking Glass https://supernoc.rogerstelecom.net/lg/)
>>> ===================================================
>>>
>>> *Query:* *tr 64.71.249.45*
>>>
>>> Type escape sequence to abort.
>>> Tracing the route to 64.71.249.45
>>>
>>>   1 64.71.255.62 0 msec 0 msec 0 msec
>>>   2 ge-4-3-0.gw02.ym.phub.net.cable.ROGERS.com (66.185.82.237) 4 msec 4 msec 4 msec
>>>   3 69.63.250.189 4 msec 4 msec 4 msec
>>>   4 69.63.250.174 4 msec 4 msec 4 msec
>>>   5  *  *  *
>>>   6  *  *  *
>>>   7  *  *  *
>>>   8  *  *  *
>>>   9  *  *  *
>>>  10  *  *  *
>>>  11  *  *  *
>>>  12  *  *  *
>>>  13  *  *  *
>>>  14  *  *  *
>>>  15  *  *  *
>>>  16  *  *  *
>>>  17  *  *  *
>>>  18  *  *  *
>>>  19  *  *  *
>>>  20  *  *  *
>>>  21  *  *  *
>>>  22  *  *  *
>>>  23  *  *  *
>>>  24  *  *  *
>>>  25  *  *  *
>>>  26  *  *  *
>>>  27  *  *  *
>>>  28  *  *  *
>>>  29  *  *  *
>>>  30  *  *  *
>>>
>>>
>>> Cheers,
>>>
>>>   - MJ
>>>
>>>
>>> -----Original Message-----
>>>> From: Grant Ridder [mailto:shortdudey123 at gmail.com]
>>>> Sent: July-10-13 10:57 AM
>>>> To: John York
>>>> Cc: nanog at nanog.org
>>>> Subject: Re: google troubles?
>>>>
>>>> Does anyone have traceroutes showing where the issues are?
>>>>
>>>> -Grant
>>>>
>>>> On Wed, Jul 10, 2013 at 7:45 AM, John York
>>>> <johny at griffintechnology.com>wrote:
>>>>
>>>> > We saw the same thing, but seems to be cleared up now. All our
>>>> > providers that routed to Google addresses in ATL had the issue. We
>>>> > have one provider that lands on Google addresses in DFW, and it was
>>>> working.
>>>> >
>>>> > ...And now I see that it isn't completely resolved. Some Google apps
>>>> > are still inaccessible via the Atlanta routes.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Jul 10, 2013 at 9:28 AM, Blair Trosper
>>>> > <blair.trosper at gmail.com
>>>> > >wrote:
>>>> >
>>>> > > Seeing lots of reports of people unable to get to many Google
>>>> services.
>>>> > >  Seems to be affecting Comcast users disproportionately.  It's fine
>>>> > > for
>>>> > me,
>>>> > > but a lot of my staff are basically out of luck...but according to
>>>> > > the Google Apps Status page, everything is fine.
>>>> > >
>>>> > > It's anecdotal, but it would seem like there's an issue based on
>>>> > > these reports.
>>>> > >
>>>> > > Oh, and this:
>>>> > > http://www.cnn.com/2013/07/10/tech/web/google-down/index.html
>>>> > >
>>>> > > Anyone know what's up?  Fiber cut?  DC outages?
>>>> > >
>>>> > > -- blair
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> >
>>>> > John York
>>>> >
>>>> > Information Technology | Network Administrator
>>>> >
>>>> > Phone: 615-399-7000 x:333
>>>> >
>>>> > Griffin Technology
>>>> > 2030 Lindell Avenue Nashville, TN  37203 USA
>>>> >
>>>> > This message and any attachments should be treated as confidential
>>>> > information of Griffin Technology, Inc.
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>
>>>