PSINet and C&W peering

Tue Jun 5 23:09:49 UTC 2001

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Leo/List:

	I know as we grow and expand into the mesh of the Internet (can I
still call it a mesh?) I will be dropping out PSINet connections
(both hosting and access - sorry engineers, but your management is a
bunch of blockheads) and not buying squat from those equally stupid
people at C&W.

	I am a huge fan of the PSINet engineering staff, I have friends in
the NYC hosting center, and all through their network...you guys keep
what's left of the company going, you should be proud.  I think
PSINet's 'ego problem' is coming back to hurt them...but their
management is STILl blind to the fact that you can't win (especially
when your ship is taking water fast) by firing rounds at the arriving
cavalry...ie, your current partners/friends in the industry.

	As far as C&W goes...if I was a customer of theirs I'd be on the
phone right now asking about my credit for network downtime.  This
HAS to count as an SLA violation - if nothing else, milk 'em for
money for being stupid.

	Screw it...I'm putting the rest of the new circuits I'm ordering
through with Savvis.  They've not disappointed me yet...delivering a
DS-3 ,and a T-1 AHEAD, that's right, AHEAD of schedule, and being
proactive...and having an IQ over 30 when answering a help line.

	Does anyone, especially those of you who are 'Internet Engineers' (I
can't claim that), have any thoughts on the changing definition of a
Tier-1 ISP?  I mean, as far as I know, it meant that to be a Tier-1
ISP 'traffic accross the 'net would touch your network at least once
in some point'....how do other companies now get away with that?

	..blah...sorry, I got off on kind of a rant there.....

<signoff>

Ralph M. Los
Asst. Vice-President, Internet Systems and Security
EnvestNet Advisory Corp.
rlos at envestnet.com
(312) 827-3945 (direct)
(312) 296-9003 (wireless w/voicemail)

- -----Original Message-----
From: Leo Bicknell [mailto:bicknell at ufp.org]
Sent: Tuesday, June 05, 2001 10:06 AM
To: Mitchell Levinn
Cc: nanog at merit.edu
Subject: Re: PSINet and C&W peering

[Disclaimer: I work for MFN/Above.net, that that for what you will.]

On Tue, Jun 05, 2001 at 09:35:20AM -0400, Mitchell Levinn wrote:
> C&W did indeed shutdown their peering connections to PSINet
> this weekend.  While there are many potential explanations
> for their actions, I have no visibility into their decision
> process.  I am disappointed with their decision to disconnect.
> PSINet continues to seek a resolution with C&W to restore normal
> connectivity in order to avoid further negative impact to both
> companies and the Internet.  Their decision is hard to understand
> based on the following:

I find this quite ironic, as PSINet decided to terminate peering
with AboveNet in November of 2000.  AboveNet was disappointed with
PSI's decision, and had worked hard to avoid the disconnection.
To this day, AboveNet is still unsure why PSI took that action,
with the damage that resulted.

The really interesting part, is the claims made below, which I will
address one by one.

> - C&W and PSINet upgraded circuits used for peering between
>   the two networks earlier this year.  C&W's recent action
>   seems inconsistent with the strategy that led to these
>   upgrades.

The same was true for PSINet and AboveNet.  AboveNet ordered multiple
OC-3 peering circuits to PSI.  The first was turned up under our
agreement.  It was not until we tried to turn up the second that
PSI decided to reevaluate AboveNet as a peer.  We were very much
caught by surprise when PSINet abruptly reverse course.

> - PSINet's recent addition of direct private peering with several
>   of C&W's transit customers relieved the peering connections
>   between the networks of a couple hundred Mbps of traffic
>   (improving connectivity overall and, undoubtedly, lowering costs
>   for those transit customers).  This is significant only because
>   C&W claims PSINet no longer has sufficient traffic to justify
>   the connections according to their published standards.  In
>   fact, PSINet's overall traffic continues to grow.

This one one of the reasons mentioned by PSINet in the disconnection
proceedings for AboveNet.  Clearly AboveNet could not be an important
peer due to the lack of traffic.  AboveNet has maintained since
day one of its existance a very open peering policy, and the low
traffic to PSI was very much in part due to the robust peering that
AboveNet had with many of PSI's downstreams.

In all fairness, the issue of traffic ratio also came up with PSINet
(so they don't come back and claim they never cared about traffic
levels, just ratio).  The ratio is equally funny though, due to 
the next point.

> - Most of the PSINet traffic previously destined for sites
>   behind C&W has alternative paths through other providers.
>   While this sounds like a generally good thing, especially given
>   the actions C&W has taken, it does make it difficult for those
>   that require certain traffic levels to be maintained consistently
>   for peering.  Specifically, C&W's customers (or C&W itself) could
>   alter "natural" traffic flow to favor (or not) various
> connections 
>   to meet their published standards (or not).  PSINet demonstrated
>   to C&W that if naturally less favorable announcements were
>   preferred, PSINet could make an almost arbitrarily large (or
>   small) amount of traffic flow between the peers.  Even so, in
>   C&W's opinion, PSINet will not be able to comply with their
>   peering policy's traffic standards.  It is gratifying to note
>   that even without C&W peering, substantially all of the
>   traffic previously flowing between PSINet and C&W continues to
>   be delivered.

This is true for any provider with a reasonable amount of traffic.
With the number of multi-homed customers behind {Sprint, C&W, UUnet,
AT&T, Genuity, Level 3, Qwest} (and possibly others) it is easy to
move hundreds of megabits of traffic between those providers while
still honoring peering agreements.  It is something PSINet ignored,
as well as C&W.

The fact is, most large ISP's can set the ration on any given
peering interface to be whatever they want, of course at the expense
of altering a ratio somewhere else.  It's like a baloon, the total
volume (ratio) is fixed, but by squeezing or pulling it you can
make it into limitless shapes.

> - At this time PSINet has not disabled the C&W peering interfaces
>   nor decommissioned any facilities.  If C&W chooses to, they can
>   re-enable interfaces on their side and bring back the
> connectivity 
>   lost between their non-transit customers and PSINet.  PSINet
>   remains open to discuss with them a new bilateral peering
>   agreement if they so choose.

How lovely. :-)  Let me try from our side "At this time AboveNet
has not disabled PSINet's peering interfaces nor decommissioned
any facilities, if PSI chooses to, they can re-enable interfaces
on their side and bring back the connectivity lost..."  AboveNet
is of course always open to discussing peering with PSINet, we hold
no grudges.

> PSINet remains committed to servicing its customers and the
> Internet with the best possible infrastructure and policies. 
> PSINet still
> maintains hundreds of peering connections with other ISPs
> throughout the world.  While posting about matters between PSINet
> and its
> peering partners is not typical, the circumstances and questions
> arising from C&W's decision required some clarification.  Hopefully
> this additional clarification helps everyone understand the current
> situation.

Now this is clearly not true.  As much as we had to swallow or
pride to do it, AboveNet took steps to make sure that even though
PSINet terminated peering, AboveNet customers experienced
uninterrupted
(if a bit rocky for a short time) connectivity to PSINet.  Our
commitment to our customers included making sure they could connect
to PSINet.  PSINet clearly doesn't care if their customers can
reach C&W.

I feel no sympathy for PSINet, as I am personally a firm believer
in "connectivity is its own reward", and I think the fairly open
AboveNet peering policy reflects that as well.  No one wins when
connectivity is broken like this, I have to believe both C&W and
PSI customers are feeling some real pain right now.  I hope this
lesson makes people think twice about terminating such peering
relationships in the future.

That said, the irony here is amazing.  PSINet has been put in the
exact same position by C&W that they decided to put AboveNet (and
possibly others, I'll let them comment on their situation) into
several months ago, and isn't even big enough to work around the
problem.  "What goes around comes around" is very often true, the
provider that was big yesterday can be small tomorrow, and could
possibly even come back to be big again.

If PSINet would like to start down the road to recovery, I suggest
they start with mending some of the relationships they chose to
sever in the past.  If a PSINet representative would like to talk
about the AboveNet situation in particular, please mail me privately
and I'll make sure you get in touch with the right people.

Also, I would like to suggest that if you are a Cable and Wireless
customer, and disagree with their current action of terminating
peering relationships (we have nasa.gov and psi.net confirmed on
Nanog, and rumors of many others in the works) I suggest you let
them know how you feel, first with your words, and if that doesn't
work with your dollars, by taking them elsewhere.  I know a lot of
people think ISP's don't listen to their customers, and sometimes
it may seem that they don't, but by and large they do try to keep
their customer base happy.

- -- 
Leo Bicknell - bicknell at ufp.org
Systems Engineer - Internetworking Engineer - CCIE 3440
Read TMBG List - tmbg-list-request at tmbg.org, www.tmbg.org

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBOx1mtnM6eMaR3AeZEQKiNACfZVnYOkpVA2xJ/Sxb64T0ObQO90EAninx
sFpZiJF8vSb9+EalU/Yafux3
=uv1R
-----END PGP SIGNATURE-----