Connectivity issue between Verizon and Amazon EC2 (NTT issue?)

Thu Jul 24 16:33:37 UTC 2014

On Tue, Jul 22, 2014 at 07:40:26PM -0700, Ray Van Dolson wrote:
> On Tue, Jul 22, 2014 at 05:29:55PM -0700, Ray Van Dolson wrote:
> > On Mon, Jul 21, 2014 at 10:54:59PM -0700, Ray Van Dolson wrote:
> > > Others appear to be having similar issues.  Seems like Verizon is
> > > pointing at AWS:
> > > 
> > > https://forums.aws.amazon.com/thread.jspa?messageID=558094
> > > 
> > > Ray
> > > 
> > > On Mon, Jul 21, 2014 at 08:56:27PM -0700, Tim Heckman wrote:
> > > > Realized I sent the reply to Roland. Apologies.
> > > > 
> > > > Here it is in full:
> > > > 
> > > > ####
> > > > 
> > > > I am seeing the same issue between AWS US-WEST 2 and Hurricane
> > > > Electric's Fremont 2 location (Linode). Looks to be deep within
> > > > Amanzon's network based on changes in latency in a simple trace
> > > > route.
> > > > 
> > > > I would provide an mtr, however my network configuration is
> > > > something mtr doesn't support.
> > > > 
> > > > Cheers!  -Tim
> > 
> > Update on this:
> > 
> > - We have a ticket open with both AWS and Verizon.
> > - AWS has responded and felt the issue was with Verizon, but notified
> >   their network team and asked them to investigate further.
> > - Nothing back from Verizon yet (anyone here have a Verizon NOC
> >   contact?)
> > 
> > In the interim, the issue persists.
> > 
> 
> Further update -- Verizon indicates that the issue is related to
> saturation on a peering link between themselves and NTT.  Verizon is
> pointing to the NTT side as the source of the saturation / congestion.
> 
> We don't have a direct customer relationship with NTT so am hoping
> someone on this list may be able to pass this information along or
> investigate on our behalf.
> 
> Ray

To close the loop on this one, Amazon made a change for us and shifted
their peering point from NTT Ashburn to NTT Dallas.

This helped out tremendously, and although we still see times where
things are "slower", it's at least not 100KB/sec slow. :)

Appreciate all the responses received.

Ray