Backbone IP network Economics - peering and transit

Patrick W.Gilmore patrick at ianai.net
Tue Apr 20 19:10:07 UTC 2004


On Apr 20, 2004, at 2:15 PM, Stephen J. Wilcox wrote:

> On Tue, 20 Apr 2004, Patrick W.Gilmore wrote:
>
>> In many, many cases, especially for smaller providers, this is a 
>> spare FE on a
>> switch which already exists.
>
> I assume Vijay meant the cost of a port for private peering, in which 
> case if
> you private with all your peers and you have a lot of small peers 
> thats going to
> be a lot of cost for a few kbps of traffic

It was Dan, not Vijay.

And clearly we are not talking about running a pair of fiber to 
everyone who has a modem's worth of traffic.  He mentioned the cost of 
the port.  I said many people have spare FEs / GEs on existing 
switches.  And if they do not, a few hundred dollars will get them one.


>>> - Operational costs such as legal review for BLPAs, NOC monitoring,
>>> troubleshooting when it flaps, putting MD5 on, etc
>>
>> These costs are frequently quoted as reasons not to peer by the larger
>> providers.
>>
>> BLPAs are only required by people who think they mean something.
>
> Well theyre a good excuse thats for certain :) But I would say they do 
> mean
> something.. if you're BigISP-A and you are peering with BigISP-B you 
> want to
> make sure that continues reliably and that means a formal arrangement. 
> Even if
> your a small ISP its worthwhile considering a formal arrangement 
> particularly
> with the larger peers to make sure they dont ditch you without some 
> good notice
> or that they will upgrade without cost if your traffic increases....

I specifically left out BigISP-*.  The complexities of peering on a 
Tier 1 network are not really describable in a single e-mail.

As for the smaller ISPs, read every peering agreement you've signed.  
They all say they can cancel with at most 30 days notice, for no 
reason, with no recourse, and nothing you can do about it.  
Furthermore, many include the ability to shut down peering if they even 
*think* you are doing something funny, and again you have no recourse.

Peering agreements are not worth anything to keep peering up.  They are 
only worth something if you are worried about the peer doing something 
like pointing default.


>> In general, Peering is a Good Thing [tm].  It increases performance, 
>> can lower
>> costs, and might even increase your network reliability.
>
> Hmm, we're fairly open on peering and have a bunch of small peers, in 
> fact most
> of our new peerings are with small peers (small is something like 
> announcing a
> single /24 and doing almost no traffic).

The second number there is important, the first is not.  There are 
peers which announce a /24 or few and have gigabits of traffic.


> We occasionally see performance problems with these small peers, where 
> they
> maybe drop the session without warning raising an alarm here or do 
> something
> screwy with their config and leak or whatever.

Nowhere was I saying it is a good idea to peer with someone who hurts 
your network.  But most of the peers, even the small ones, can keep 
their network stable.


> They also tend to only have one connection, this forces how we route 
> traffic to
> them, as we're in the process of expanding I really want to have 
> multiple equal
> paths so that we can be sure the traffic is taking the best way to 
> them.

Perfectly valid concern.  Which is why I specifically told people to 
find out who would peer with them before paying to go to a peering 
point.  Don't count your chickens until they're hatched and all that. 
:)


> My summary of these points is that I'm seriously considering what our 
> policy
> will be in the future and for good reason (altho it will undoubtedly 
> continue to
> be fairly relaxed).

And I see nothing you mentioned which in any way goes against what I 
was saying.  Your particular situation is very different than the next 
networks, as the next networks is unique to that network, etc.  But 
that doesn't make peering bad.


>> If your monthly costs are lower with peering than transit alone, it is
>> probably a good idea to peer and ignore the NOC costs.
>
> In some instances I'm willing to pay more for a connection (eg paid 
> peering or
> costs of backbone circuits) to ensure I'm receiving quality.

It is nice to ensure quality.  But if quality is your primary goal, 
then directly peering with a network will give you better "quality" 
from an end user (read "paying customer") PoV than transit in most 
cases.  Extra latency is usually not viewed as better quality.

If you are worried about the connection being flaky, well, like I said, 
don't peer with flaky networks.

Besides, most small to medium guys have enough headroom on their 
transit connections to take down many of their peers and push it over 
transit without congestion.


> There are a couple other issues not raised...
>
> One is the cost on the router in terms of memory and cpu of 
> maintaining such a
> large number of sessions (usually less of an issue with your big 
> multiprocessor
> routers)

Agreed.  But since we are not talking to the one-T1-ISP (which I also 
said would not fit the model), people probably have enough CPU to 
handle a few extra BGP sessions.

If not, well, another cost to consider before peering.


> The other is our new hot topic of security, not sure if anyone has 
> thought of
> this yet (or how interesting it is) but the nature of the bgp attack 
> means that
> if you can view a BGP session you can figure things about a peer that 
> would
> otherwise be hidden from you in particular the port numbers in use.. 
> and I'm not
> entirely clear on the details but it sounds like when you hit the 
> first session,
> you can take the rest out very easily.

Riiiiiiiiiiiiiiiiiiiight.


> We cant take BGP out of band (yet!), perhaps we can keep it better 
> hidden from
> view tho..

Good idea.

Get right on that, would you? :)

-- 
TTFN,
patrick




More information about the NANOG mailing list