Verizon connectivity issues

Christopher Morrow morrowc.lists at gmail.com
Thu Aug 28 02:51:49 UTC 2014


On Mon, Aug 25, 2014 at 1:54 PM, Ryan Ruuska <ryanruuska+nanog at gmail.com> wrote:
> Hello all,
>
> We have heard from many of our customers in the Northeast region,
> specifically PA, MD, and VA who have difficulty connecting to our website

there's probably vz customer folk on-list, maybe 'what should they
test for you' would be nice? :)

> (very slow loading times).  We have noticed that if our data center
> provider specifically routes our outbound traffic over Hurricane Electric
> rather than XO Communications, that connectivity is restored to normal for
> our customers.  The HE path goes from Salt Lake City to Denver where it is
> handed off to TeliaSonera, and they send it to Chicago where it gets handed
> off to Verizon.  This works great and generally has a ping response of
> 20-25ms less than the bad route as posted below from one of our servers:
>
> Tracing route to pool-108-52-xxx-xxx.phlapa.fios.verizon.net
>  [108.52.xxx.xxx]
> over a maximum of 30 hops:
>
>   1    <1 ms    <1 ms    <1 ms  172.xxx.xxx.xxx (Internal IP)
>   2    <1 ms    10 ms    <1 ms  209-41-xxx-xxx.c7dc.com [209.41.xxx.xxx]
>   3    <1 ms    <1 ms    <1 ms  e1-5.rt08.gp1.c7dc.com [192.41.0.97]
>   4     2 ms     1 ms     1 ms  ip65-46-51-113.z51-46-65.customer.algx.net
>  [65.46.51.113]
>   5    19 ms    18 ms    19 ms  vb1611.rar3.sanjose-ca.us.xo.net
>  [216.156.0.5]
>   6    21 ms    18 ms    18 ms  207.88.14.226.ptr.us.xo.net [207.88.14.226]
>   7    18 ms    18 ms    18 ms  206.111.6.122.ptr.us.xo.net [206.111.6.122]
>  >>>> Verizon owned device with an XO IP address
>   8    87 ms    86 ms    87 ms  b100.phlapa-lcr-22.verizon-gni.net
> [130.81.209.187]
>  >>>> Next hop goes from CA to PA, probably just hidden routing info
>   9     *        *        *     Request timed out.
>  10    88 ms    89 ms    90 ms  pool-108-52-xxx-xxx.phlapa.fios.verizon.net
>  [108.52.xxx.xxx]
>
> As you can see, XO sends this traffic from Salt Lake City to San Jose where
> it is handed off to Verizon.  We've worked with XO and determined that
> there are no connectivity issues along their path, which seems to point to
> an issue within Verizon's network.  I have a couple of tickets open with
> Verizon at the moment on behalf of some of our customers, but we have had
> trouble breaking through Tier 1.
>
> Our first thought was saturation of the peering point, but XO states that
> the link between hops 6 and 7 is using 5Gbps out of 20Gbps capacity, and
> the latency on hop 7 (the Verizon owned device with an XO IP) seems normal
> whenever we test.
>
> Any Verizon engineers out there willing to take a quick peek at this route
> for any obvious issues?  It would be a lot easier for us if Verizon
> provided Looking Glass servers, but alas...

they have route data sent to routeviews at least...


More information about the NANOG mailing list