XO blocking individual IP's

Tue Nov 8 04:34:23 UTC 2011

Oh yes!  Good lord I about went insane with this.  I was working with a customer single homed to cBeyond.  I spent 3 hours on the phone with cBeyond to figure out what was going on, it looks like a broken route.  Come to find out it was an XO "security null".  The engineer on the phone from cBeyond said to me "Well, I have learned 2 things today.  1, XO nulls for 'security purposes' at random.  2, I am no longer shocked by any ridiculous policy I will ever come across again."

In this case majority traffic was going from cBeyond to anywhere (via XO) and being eaten, however it was VERY tough to diagnose as all parties involved assumed this would not be occurring between source and destination without good public documentation or at least any record of this happening to someone else.  Also I guess we all assumed that major bandwidth players don't filter anything.

I personally think its good on paper, but very bad real life until there is a way to notify the end customer of the violation quickly.  This issue literally took 3 full weeks to figure out what was going on.  Yes this works great in a colo datacenter as you have the customer contact info (hopefully).  But in the case where my customers provider was having the IP filtered by their transit it was hell to diagnose.  In my case the customer had a single infected machine that was making outbound connections on TCP3389 in the range of about 100 connections every 5 minutes and because of this was entirely being "security nulled".

Blake

-----Original Message-----
From: clayton at haydel.org [mailto:clayton at haydel.org] 
Sent: Monday, November 07, 2011 7:43 PM
To: nanog at nanog.org
Subject: XO blocking individual IP's

I'm hoping someone has had the same experiences, and is further toward a resolution on this than I am. About 6 months ago, we noticed that XO was blackholing one specific IP out of a /24.  Traces to that IP stopped on XO's network, traces to anything else out of the block went through fine.
XO finally admitted that they had a new security system that identifies suspicious traffic and automatically blocks the IP for 30 minutes.  We had to get the IP in question "whitelisted" by their security guys.  The traffic was all legit, it was just on a high port # that they considered suspicious.

There have several more cases like this, and XO has not been forthcoming with information. We're either looking to be exempted from this filtering or at least get a detailed description of how the system works.  I'm not sure how they think this is acceptable from a major transit provider.
Anybody else had similar problems?

Clayton Haydel