NTT communications horrible routing, unresponsive NOC

Wed Mar 23 23:39:55 UTC 2016

Hi all,

I've been trying to get this issue resolved for the entire day now, but NTT
has been pretty unreceptive here.

We're announcing a large prefix for a client across our network, and we
discovered some insanely high latency.

After tracking down the issue, we determined it to be something wrong with
NTT at their Seattle location. We anycast this prefix, but no matter where
in the world traffic is originating from, it's going to Seattle and then to
Atlanta. Example: Rotterdam in the Netherlands routes from Europe -> east
coast -> west coast Seattle -> los angeles -> atlanta. The gist of it is
that something is seriously messed up at NTT in Seattle.

We contacted our transit provider to try and carry the issue upstream, and
what they told us was

Sorry for delay, I've asked NTT to clear the more specific for this one as
well.
The problem seems to be a bug on the NTT side which keeps stale routes in
the routing table for more specifics ( at random ).
If you have more routes affected please notify us of the routes and I will
ask them to clear the routing table for these routes.
NTT is working on this with their vendor to get this resolved as soon as
possible.

I had spoken to a sales rep for NTT a few weeks prior, and they assured me
that the NOC was top notch, and that all routes were redundant, and they
guaranteed less than 50ms in the US, and all kinds of marketing. However,
it looks like it's all marketing - for this entire day this router has been
causing tons of issues for our clients.

No-exporting it to NTT does not even solve the problem, as NTT's router in
Seattle apparently just decides to keep random small prefixes in it,
causing traffic to go there.

At a loss as to what to do now, since their NOC isn't receptive. Anyone
have someone I can contact off-list to get this issue resolved? It's
especially frustrating because the problem absolutely cannot be resolved on
our end, even with a no-export since NTT is keeping the routes in their
router.