Backbone IP network Economics - peering and transit

Tue Apr 20 15:52:16 UTC 2004

On Apr 20, 2004, at 10:32 AM, Daniel Golding wrote:

> On 4/20/04 1:34 AM, "Michel Py" <michel at arneill-py.sacramento.ca.us> 
> wrote:
>
>>> Patrick W.Gilmore wrote:
>>> Unless they have cheap access to a free NAP (TorIX, SIX, etc.),
>>> transit, even at higher prices, is probably be the best /
>>> cheapest way to reach the Internet.
>>
>> This is true, but there are plenty of other opportunities for peering,
>> such as: both parties buy DS-3 class transit from the same tier-2 or
>> even maybe tier-3 provider in a colo (which will likely be a BFM, 
>> other
>> problem) not a formal IX. In other words, peering in an IX does cost
>> money, but peering at a colo might not, as these messy colos are 
>> mostly
>> unmanaged and nobody cares about that 25ft cross-over cable :-)
>>
>> Michel.
>>
>>
>
> This is a classical mistake. Peering always costs money and its never 
> free.

Maybe "Free" is the wrong word.  Perhaps "No additional cost over 
<transit/whatever>".

Or, for those of us who think that the time it takes to plug a patch 
cable into an unused switch port and do some configuration changes are 
irrelevant, maybe "free" is the right word.

Either way, it is not NEARLY as bad as you or many other people make it 
out to be.  Allow me to explain....

> The question is, how much, and is it cheaper than transit?
>
> Costs incurred in peering:
>
> - Port Costs (capex)

Pthhhhhhhhh.  In many, many cases, especially for smaller providers, 
this is a spare FE on a switch which already exists.

For mid-sized providers, it is frequently a spare GE port on an 
existing switch, which means perhaps $500 for GBIC or something.

For large providers, there is a cost here.  But large providers are a 
different beast, and there is no way a simple e-mail could possibly 
capture the complexities implicit in peering between Very Large 
Providers.  So we'll let them figure out their own costs.

> - A share of a router's backplane capacity corresponding to the port

Irrelevant.  The traffic has to go somewhere, if it does not go out the 
peering port, it will go out transit, but it is definitely going across 
the router's backplane.

A better thing to put here would be possible use of a router which 
would not be used.  Specifically, if I get a bit in a POP which has 
transit, I do not have to use the router out at the Peering Point.  But 
how many people have router backpanes which are saturated?  At worst 
you are running out of slots for ports in most cases.  (Remember, we 
left the really big providers to their own devices.)

> - Cross connect costs (one time or recurring)

Largely irrelevant - if you are really going to go out-of-biz for a 
$150/Month x-conn, you have bigger problems.

> - Operational costs such as legal review for BLPAs, NOC monitoring,
> troubleshooting when it flaps, putting MD5 on, etc

These costs are frequently quoted as reasons not to peer by the larger 
providers.  Strangely enough, if you are not a Tier 1 (or hoping to be 
a Tier 1), peering sessions are usually "set up and forget".  Networks 
who have 10s of gigabits of traffic but are not looking for reasons to 
deny peering requests see nearly no cost in these (especially compared 
to the overall cost of running the network).

BLPAs are only required by people who think they mean something.  
Putting on MD5 is a bug/unique situation, which affect peering perhaps 
once every half-decade or so.  Most small and mid-sized providers can 
handle the "NOC monitoring, trouble shooting, etc." with 
single-digit-hours a month, max.  And sometimes that time is handled by 
people who are sitting on their ass waiting for something to break 
anyway.  (Read "sunk cost".)

So, unless you are looking for reasons to *not* peer, these are mostly 
BS.

> - Administration

Think we covered this one.

> - Public Switch costs

This is a cost and should be considered.  Unless, of course, you are at 
TorIX, SIX, or any of the other very fine free NAPs available.  Or if 
you can x-conn between your rack and someone else's rack in the same 
colo facility without going to a public switch.  Or if you are in a FR 
or ATM cloud with other providers and can get uber-cheap PVCs between 
your routers with no additional hardware and a simple configuration 
change.  Or....

I think you get the point. :)

> It is difficult to defend peering strategies today unless your network 
> is of
> a fairly significant size (gigabits of traffic) and you are collocated 
> in an
> advantageous location(s). Otherwise, low cost transit is hard to beat.

I think you mean "or you are colocated in an advantageous location", 
not "and".  If I am in 151 Front street, for a small one-time fee, I 
can connect to TorIX.  The amount of the fee and the time it takes to 
set up peering is probably in the noise, even for a relatively small 
provider.

Obviously if my entire traffic fits on a T1, things might be different, 
but I do not need anywhere near a gigabit of traffic to justify 
peering.  You are probably at least an order of magnitude off.

In general, Peering is a Good Thing [tm].  It increases performance, 
can lower costs, and might even increase your network reliability.  But 
all the "other" things (e.g. performance benefits) are probably 
nice-to-have, not requirements.  I'd look at the money.

If you can break even or better, peering is probably a good idea.  Most 
of the things analysts and Tier 1 providers talk about with peering 
costs (legal costs with contracts, managing sessions, NOC time, etc.) 
are mostly irrelevant, especially to anyone without a stock symbol and 
the related overhead of large corporations.  (In many cases, they are 
irrelevant even to companies with those things.)  So look at your real 
costs, and real savings:

Hard costs - How much is the NAP connection?  How much is the line to 
the NAP?  How much for a router at the NAP?  (Etc.)

Then look at your benefits - Who will peer with me?  How much traffic 
can I dump to them?  (Etc.)

Add up all the costs in the first set of questions (one time and 
recurring), subtract your transit cost times the amount of traffic you 
will save, and see if it is positive or negative.  Don't forget to 
factor in things like any additional cost you incur by having less 
traffic to commit to a transit provider.  (For instance, if you are 
using tiered pricing, will dumping traffic to peers bring you down to a 
lower tier, and therefore a higher $/Mbps?)

If your monthly costs are lower with peering than transit alone, it is 
probably a good idea to peer and ignore the NOC costs.  Everyone's 
situation is different, but don't put too much stock in things like the 
cost of a good BLPA. :)

-- 
TTFN,
patrick