Cogent <-> Verizon peering congestion

Tue Feb 4 23:44:14 UTC 2014

Hello,

For the last several months, we have been tracking a congestion issue 
between Cogent <-> Verizon

  Host Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. router.garlic.com 0.0%    29    0.3   6.1   0.2 160.6  29.7
  2. vl203.mag03.sfo01.atlas.cogentco.com 0.0%    29    2.2   8.1   2.1 
161.1  29.5
  3. te0-0-0-14.ccr22.sfo01.atlas.cogentco.com 0.0%    29    2.9   2.7   
2.4   3.6   0.2
  4. be2165.ccr22.sjc01.atlas.cogentco.com 0.0%    29    4.1   4.0   
3.7   4.8   0.2
  5. be2047.ccr21.sjc03.atlas.cogentco.com 0.0%    29    4.5   4.7   
4.3   5.5   0.3
  6. verizon.sjc03.atlas.cogentco.com 22.2%    28  169.3 171.5 168.1 
193.5   6.9
  7. so-1-0-0-0.SJC01-CORE-RTR2.verizon-gni.net 37.0%    28  205.8 180.6 
171.6 271.6  24.8
  8. A12-0-135.SNFCCA-DSL-01.verizon-gni.net 33.3%    28  172.3 177.5 
171.7 250.8  18.3
  9. pool-71-116-122-235.snfcca.btas.verizon.net 25.0%    28  197.9 
197.6 195.5 199.2   0.8

We have smokeping's from our side showing 30%+ packet loss from us 
(AS4307) to Verizon.

All I have gotten from Cogent is a canned response:

---
The latency and/or packet loss that you are experiencing to this 
destination is due to occasional high traffic with Verizon. We have 
repeatedly requested augments to these congestion points and hope 
Verizon will comply soon.  While this has been escalated internally to 
the CEO level, we encourage you to also contact Verizon customer support 
with your concerns and complaints.  Their delay is a major impediment to 
internet traffic overall and contrary to net neutrality requirements.  
Our peering engineers will continue to address this on a daily basis 
until resolved.
---

It seems to have gotten a lot worse in recent days, to the point where 
we have customers who are trying to access us from Verizon's network 
(i.e. they have Verizon DSL, or via Verizon 3G/4GLTE) complaining they 
are having a very hard to checking their email, etc.

Has anyone else been experiencing these issues?  Or does anyone have 
more information that what Cogent provided me in their canned statement?

-Bobby