It worked! Huge Thanks Re: verizon trouble ticket NJ DQ04PWR9 -- is verizon blocking FLOKsociety.org by accident or on purpose?

Fri Oct 4 20:48:46 UTC 2013

Really glad i posted  take a boqand thank you again!
=============================================================
The COOK Report on Internet Protocol, (PSTN) 609 882-2572 
Back Issues: http://www.cookreport.com/index.php?option=com_docman&task=cat_view&gid=37&Itemid=61  
 Cook's Collaborative Edge Blog 
 http://www.cookreport.com/wp/
  Subscription info: http://www.cookreport.com/index.php?option=com_content&view=article&id=54&Itemid=65
=============================================================

On Oct 4, 2013, at 3:50 PM, "Moore, Matthew S" <mscott.moore at verizon.com> wrote:

> Thanks eric...scott
> 
> From: Sieg, Eric W
> Sent: Friday, October 4, 2013 3:14 PM
> To: Christopher Morrow
> Cc: Moore, Matthew S; Gordon Cook; Young, David E
> Subject: RE: verizon trouble ticket NJ DQ04PWR9 -- is verizon blocking FLOKsociety.org by accident or on purpose?
> 
> Everything is complete, all associated prefix-lists for this customer have been updated. 
>  
> Thank you for choosing Verizon,
>  
> Eric Sieg
> IP Non Managed Operations Support | IP/DNS/DNE Administration
> Tel: 248 728 5294 | Vnet: 443-5294
>  
> <image001.gif>
> <image002.jpg>  <image003.jpg>  <image004.jpg>  <image005.jpg>
>  
> From: christopher.morrow at gmail.com [mailto:christopher.morrow at gmail.com] On Behalf Of Christopher Morrow
> Sent: Friday, October 04, 2013 3:01 PM
> To: Sieg, Eric W
> Cc: Moore, Matthew S; Gordon Cook; Young, David E
> Subject: Re: verizon trouble ticket NJ DQ04PWR9 -- is verizon blocking FLOKsociety.org by accident or on purpose?
>  
>  
>  
> 
> On Fri, Oct 4, 2013 at 2:59 PM, Christopher Morrow <morrowc.lists at gmail.com> wrote:
>  
>  
> 
> On Fri, Oct 4, 2013 at 2:54 PM, Sieg, Eric W <eric.sieg at verizon.com> wrote:
> I’ll get it added to their other sites and follow up with them to let them know.  Give me about 20.
>  
>  
> sweet, thanks!
>  
> forgot, from a fios customer perspective:
>  2  G0-9-4-6.WASHDC-LCR-22.verizon-gni.net (130.81.183.118)  10.434 ms  6.308 ms  3.699 ms
>  3  ae6-0.RES-BB-RTR1.verizon-gni.net (130.81.199.146)  5.45 ms ae5-0.RES-BB-RTR1.verizon-gni.net (130.81.209.222)  3.375 ms ae2-0.RES-BB-RTR1.verizon-gni.net (130.81.199.138)  4.592 ms
>  4  0.ae4.XL2.IAD8.ALTER.NET (152.63.8.125)  8.32 ms  3.457 ms  5.700 ms
>  5  0.xe-9-3-0.GW9.IAD8.ALTER.NET (152.63.36.34)  5.820 ms 0.xe-9-0-0.GW9.IAD8.ALTER.NET (152.63.36.18)  7.311 ms 0.xe-11-2-1.GW9.IAD8.ALTER.NET (152.63.42.2)  6.568 ms
>  6  telefonica-gw.customer.alter.net (152.179.50.114)  3.767 ms  4.74 ms  5.569 ms
>  7  Te0-7-0-5-grtmiana4.red.telefonica-wholesale.net (94.142.126.182)  48.345 ms  46.997 ms Xe4-1-6-0-grtmiana2.red.telefonica-wholesale.net(94.142.123.145)  46.154 ms
>  8  176.52.251.197 (176.52.251.197)  44.747 ms 176.52.251.189 (176.52.251.189)  74.746 ms 176.52.249.241 (176.52.249.241)  49.805 ms
>  9  176.52.252.66 (176.52.252.66)  99.787 ms  98.471 ms  97.103 ms
> ^C
>  
> looks like it works now.
>  
>  
> Thank you for choosing Verizon,
>  
> Eric Sieg
> IP Non Managed Operations Support | IP/DNS/DNE Administration
> Tel: 248 728 5294 | Vnet: 443-5294
>  
> <image001.gif>
> <image002.jpg>  <image003.jpg>  <image004.jpg>  <image005.jpg>
>  
> From: Moore, Matthew S 
> Sent: Friday, October 04, 2013 2:43 PM
> To: Christopher Morrow; Gordon Cook
> Cc: Young, David E; Sieg, Eric W
> Subject: RE: verizon trouble ticket NJ DQ04PWR9 -- is verizon blocking FLOKsociety.org by accident or on purpose?
>  
> Hi Chris,
> 
>  
> 
> Uncloaking..... ;-)
> 
>  
> 
> It does look as though it's a matter of an un-updated prefix-filter for telefonica.  The existing filter allows a bunch of /24's
> 
> close to the address in question but 200.10.150.0/24 is not among them, whereas telefonica is announcing it to us. (shown, hidden, below)
> 
>  
> 
> scottm at GW9.IAD8> show route receive-protocol bgp 152.179.50.114 hidden | match 200.10.150
> 
>   200.10.150.0/24         152.179.50.114       0                  12956 12956 12956 12956 12956 12956 12956 19169 27947 28027 I
> 
>  
> 
> I added the filter, and I'm cc'ng Eric in Customer Support to let him know.  It will make sense to update it across all 12956 sessions, as I only did this one… no longer hidden… 
>  
> 
>  
> 
> scottm at GW9.IAD8> show route receive-protocol bgp 152.179.50.114 | match 200.10.150
> 
> * 200.10.150.0/24         152.179.50.114       0                  12956 12956 12956 12956 12956 12956 12956 19169 27947 28027 I
> 
>  
> 
>  
> 
> Regards...Scott
> 
>  
> 
> -----Original Message-----
> From: christopher.morrow at gmail.com [mailto:christopher.morrow at gmail.com] On Behalf Of Christopher Morrow
> Sent: Friday, October 04, 2013 1:39 PM
> To: Gordon Cook
> Cc: Young, David E
> Subject: Re: verizon trouble ticket NJ DQ04PWR9 -- is verizon blocking FLOKsociety.org by accident or on purpose?
> 
>  
> 
> I always go with my standby: scott moore ... he can uncloak from bcc land if he's able/willing/etc to help in this case.
> 
>  
> 
> -chris
> 
>  
> 
> On Fri, Oct 4, 2013 at 1:37 PM, Gordon Cook <cook at cookreport.com> wrote:
> 
> > again thank you very very much.  I have to leave routes huit for  root canal but i do expect to be back  can anyone nominate someone at verizon who would do to checking with telefonca?
> 
> > 
> 
> > david young is the only guy i know.
> 
> > 
> 
> > But question now is how to rattle a cage at verizon and possible
> 
> > telefonica - I will also send word of this trouble shooting to the
> 
> > folk in ecuador
> 
> > =============================================================
> 
> > The COOK Report on Internet Protocol, (PSTN) 609 882-2572 Back Issues:
> 
> > http://www.cookreport.com/index.php?option=com_docman&task=cat_view&gi
> 
> > d=37&Itemid=61
> 
> >  Cook's Collaborative Edge Blog
> 
> >  http://www.cookreport.com/wp/
> 
> >   Subscription info:
> 
> > http://www.cookreport.com/index.php?option=com_content&view=article&id
> 
> > =54&Itemid=65
> 
> > =============================================================
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > On Oct 4, 2013, at 1:22 PM, Christopher Morrow <morrowc.lists at gmail.com> wrote:
> 
> > 
> 
> >> On Fri, Oct 4, 2013 at 1:15 PM, Gordon Cook <cook at cookreport.com> wrote:
> 
> >>> Hi chris
> 
> >>> 
> 
> >>> really appreciate the help from ALL you guys
> 
> >> 
> 
> >> sure thing.
> 
> >> 
> 
> >>> does what you just said mean that non reachability for version customer  may mean a config problem for a small bloc and not something intentional??
> 
> >> 
> 
> >> that's probably hard to say... I do know that:
> 
> >> 
> 
> >> 5  0.xe-9-2-0.GW9.IAD8.ALTER.NET (152.63.36.30)  6.457 ms  6.821 ms
> 
> >> 0.xe-11-2-1.GW9.IAD8.ALTER.NET (152.63.42.2)  5.932 ms
> 
> >> 6  telefonica-gw.customer.alter.net (152.179.50.114)  7.182 ms  5.433
> 
> >> ms  5.431 ms
> 
> >> 7  Xe4-1-6-0-grtmiana2.red.telefonica-wholesale.net (94.142.123.145)
> 
> >> 47.271 ms  48.381 ms Te0-7-0-5-grtmiana4.red.telefonica-wholesale.net
> 
> >> (94.142.126.182)  50.290 ms
> 
> >> 
> 
> >> 
> 
> >> telefonica is a 'customer' not a 'peer' of 701, based on their
> 
> >> connectivity. This means that telefonica has to tell 701: "Yo, I need
> 
> >> you to accept routes for x, y, z, ktnxbi!'
> 
> >> 
> 
> >> knowing the normal (well, 5yrs ago) config for customers of 701, I
> 
> >> don't expect there'd be special handling of this prefix either... so
> 
> >> I suspect either no one told 701 to accept this, or telefonica
> 
> >> pouched it at the handoff :(
> 
> >> 
> 
> >> If they didn't do that, these routes wouldn't be accepted.
> 
> >> If telefonica botched some filter on their side (see the comment
> 
> >> about prepending).
> 
> >> 
> 
> >> It's fairly sure though that the prefix isn't 'blackholed'... since
> 
> >> it's everything in the /22 not just the local /32 or /24. Bill
> 
> >> Herrin's on target as well, it's really hard to say from here :)
> 
> >> 
> 
> >> -chris
> 
> >> 
> 
> >>> 
> 
> >>> On Oct 4, 2013, at 12:54 PM, Christopher Morrow <morrowc.lists at gmail.com> wrote:
> 
> >>> 
> 
> >>>> err.. nothing in the /24 is reachable from 701's perspective (so it
> 
> >>>> seems)... so I'd suspect that there's a routing problem with the
> 
> >>>> /24, in fact the surrounding /24's also seem to be having the same problem.
> 
> >>>> 
> 
> >>>> On Fri, Oct 4, 2013 at 12:42 PM, Miles Fidelman
> 
> >>>> <mfidelman at meetinghouse.net> wrote:
> 
> >>>>> Also inaccessible from FIOS Boston:
> 
> >>>>> new-host-2:~ mfidelman$ traceroute floksociety.org traceroute to
> 
> >>>>> floksociety.org (200.10.150.169), 64 hops max, 52 byte packets
> 
> >>>>> 1  wireless_broadband_router (192.168.1.1)  1.534 ms  0.853 ms
> 
> >>>>> 0.724 ms
> 
> >>>>> 2  l100.bstnma-vfttp-84.verizon-gni.net (96.252.37.1)  7.619 ms
> 
> >>>>> 6.855 ms
> 
> >>>>> 7.304 ms
> 
> >>>>> 3  200.10.150.169 (200.10.150.169)  10.482 ms !N *^C
> 
> >>>>> 
> 
> >>>>> But just fine from our datacenter via xo.net.  And the web server
> 
> >>>>> is up - at least to a text browser (Lynx).
> 
> >>>>> 
> 
> >>>>> Also via Verizion cell network (Boston area).
> 
> >>>>> 
> 
> >>>>> Some kind of routing table glitch or peering issue, perhaps?
> 
> >>>>> 
> 
> >>>>> 
> 
> >>>>> 
> 
> >>>>> William Herrin wrote:
> 
> >>>>>> 
> 
> >>>>>> On Fri, Oct 4, 2013 at 12:09 PM, Gordon Cook <cook at cookreport.com> wrote:
> 
> >>>>>>>> 
> 
> >>>>>>>> traceroute to floksociety.org (200.10.150.169), 64 hops max, 72
> 
> >>>>>>>> byte packets
> 
> >>>>>>>> 1  192.168.1.1 (192.168.1.1)  0.759 ms  0.309 ms  0.357 ms
> 
> >>>>>>>> 2  l100.cmdnnj-vfttp-26.verizon-gni.net (98.110.50.1)  36.778
> 
> >>>>>>>> ms
> 
> >>>>>>>> 17.508 ms  7.316 ms
> 
> >>>>>>>> 3  * g0-3-4-5.cmdnnj-lcr-21.verizon-gni.net (130.81.184.119) 
> 
> >>>>>>>> 6.482 ms !N *
> 
> >>>>>>>> 4  * * g0-3-4-5.cmdnnj-lcr-21.verizon-gni.net (130.81.184.119) 
> 
> >>>>>>>> 7.101 ms !N
> 
> >>>>>>>> 5  * g0-3-4-5.cmdnnj-lcr-21.verizon-gni.net (130.81.184.119) 
> 
> >>>>>>>> 9.239 ms !N *
> 
> >>>>>>>> 6  g0-3-4-5.cmdnnj-lcr-21.verizon-gni.net (130.81.184.119) 
> 
> >>>>>>>> 6.823 ms !N *  8.846 ms !N
> 
> >>>>>> 
> 
> >>>>>> Inaccessible via FIOS Washington DC too:
> 
> >>>>>> 
> 
> >>>>>> traceroute -T -p 80 200.10.150.169 traceroute to 200.10.150.169
> 
> >>>>>> (200.10.150.169), 30 hops max, 40 byte packets
> 
> >>>>>> 1  L300.WASHDC-VFTTP-91.verizon-gni.net (173.73.47.1)  1.804 ms
> 
> >>>>>> 1.595 ms  1.562 ms
> 
> >>>>>> 2  G0-6-4-7.WASHDC-LCR-22.verizon-gni.net (130.81.216.250)  5.321
> 
> >>>>>> ms !N
> 
> >>>>>> * *
> 
> >>>>>> 
> 
> >>>>>> Correctly accessible via Cox, Qwest, Sprint and others, but the
> 
> >>>>>> network path is really slow and really long.
> 
> >>>>>> 
> 
> >>>>>> The border is consistently with telefonica-wholesale.net and then
> 
> >>>>>> telconet.net. Beyond the border there are badly behaving routers,
> 
> >>>>>> including ones configured with RFC 1918 addresses. The
> 
> >>>>>> addressable routers are reachable via Verizon, just not the last hop.
> 
> >>>>>> 
> 
> >>>>>> traceroute -T -p 80 200.10.150.169 traceroute to 200.10.150.169
> 
> >>>>>> (200.10.150.169), 30 hops max, 60 byte packets
> 
> >>>>>> 1  sark.dirtside.com (70.182.189.216)  0.708 ms  0.689 ms  0.569
> 
> >>>>>> ms
> 
> >>>>>> 2  10.1.192.1 (10.1.192.1)  9.957 ms  9.874 ms  9.725 ms
> 
> >>>>>> 3  ip68-100-3-49.dc.dc.cox.net (68.100.3.49)  9.631 ms  9.507 ms 
> 
> >>>>>> 9.424 ms
> 
> >>>>>> 4  ip68-100-3-113.dc.dc.cox.net (68.100.3.113)  9.310 ms  9.226
> 
> >>>>>> ms
> 
> >>>>>> 9.140 ms
> 
> >>>>>> 5  mrfddsrj02gex070002.rd.dc.cox.net (68.100.0.145)  9.111 ms 
> 
> >>>>>> 9.019 ms  8.929 ms
> 
> >>>>>> 6  68.1.4.139 (68.1.4.139)  8.791 ms *  5.981 ms
> 
> >>>>>> 7  209.48.42.61 (209.48.42.61)  5.748 ms  11.361 ms  10.948 ms
> 
> >>>>>> 8  vb2000d2.rar3.washington-dc.us.xo.net (207.88.13.66)  58.454
> 
> >>>>>> ms
> 
> >>>>>> 52.415 ms  52.421 ms
> 
> >>>>>> 9  te-3-0-0.rar3.atlanta-ga.us.xo.net (207.88.12.9)  60.543 ms
> 
> >>>>>> 60.397 ms  60.378 ms
> 
> >>>>>> 10  te-3-0-0.rar3.dallas-tx.us.xo.net (207.88.12.2)  58.211 ms 
> 
> >>>>>> 58.407 ms  58.392 ms
> 
> >>>>>> 11  * * *
> 
> >>>>>> 12  206.111.5.226.ptr.us.xo.net (206.111.5.226)  53.378 ms 
> 
> >>>>>> 49.080 ms
> 
> >>>>>> 47.435 ms
> 
> >>>>>> 13  Xe-8-1-0-0-grtmiabr3.red.telefonica-wholesale.net
> 
> >>>>>> (94.142.125.54)
> 
> >>>>>> 76.006 ms Xe8-0-2-0-grtmiabr4.red.telefonica-wholesale.net
> 
> >>>>>> (94.142.119.38)  60.181 ms
> 
> >>>>>> Xe13-1-4-0-grtmiabr4.red.telefonica-wholesale.net
> 
> >>>>>> (213.140.43.109)
> 
> >>>>>> 125.888 ms
> 
> >>>>>> 14  Te-0-2-0-0-grtmiana4.red.telefonica-wholesale.net
> 
> >>>>>> (94.142.119.233)
> 
> >>>>>> 67.105 ms Te0-1-0-0-grtmiana4.red.telefonica-wholesale.net
> 
> >>>>>> (213.140.37.77)  63.435 ms
> 
> >>>>>> Xe5-1-8-0-grtmiana2.red.telefonica-wholesale.net (213.140.36.89)
> 
> >>>>>> 141.873 ms
> 
> >>>>>> 15  Xe9-3-0-0-gramiana4.red.telefonica-wholesale.net
> 
> >>>>>> (94.142.126.197)
> 
> >>>>>> 62.450 ms 176.52.249.245 (176.52.249.245)  66.665 ms
> 
> >>>>>> 176.52.249.241
> 
> >>>>>> (176.52.249.241)  64.668 ms
> 
> >>>>>> 16  176.52.252.66 (176.52.252.66)  118.619 ms  118.057 ms 
> 
> >>>>>> 117.934 ms
> 
> >>>>>> 17  * * *
> 
> >>>>>> 18  * * *
> 
> >>>>>> 19  * * *
> 
> >>>>>> 20  host-186-5-116-193.telconet.net (186.5.116.193)  122.586 ms
> 
> >>>>>> 120.967 ms  115.040 ms
> 
> >>>>>> 21  host-186-101-89-42.telconet.net (186.101.89.42)  122.801 ms
> 
> >>>>>> 125.164 ms  119.520 ms
> 
> >>>>>> 22  * * *
> 
> >>>>>> 23  200.10.150.169 (200.10.150.169)  253.710 ms  246.684 ms 
> 
> >>>>>> 244.845 ms
> 
> >>>>>> 
> 
> >>>>>> 
> 
> >>>>>> 
> 
> >>>>>> 
> 
> >>>>>> 
> 
> >>>>> 
> 
> >>>>> 
> 
> >>>>> --
> 
> >>>>> In theory, there is no difference between theory and practice.
> 
> >>>>> In practice, there is.   .... Yogi Berra
> 
> >>>>> 
> 
> >>>>> 
> 
> >>>> 
> 
> >>> 
> 
> > 
> 
>  
>  