Coop Peering Fabric??

Patrick W. Gilmore patrick at
Mon Aug 11 22:37:58 CDT 2008

As a big ra-ra guy around peering, I thought this might be  
interesting, but I do not think I agree with the numbers.

On Aug 11, 2008, at 11:15 PM, Deepak Jain wrote:

> Given Cogent (and others) recent pursuit of sub $4/mb/s transit...  
> and the relatively flat cost of a "paid" peering fabric (even at  
> 10G) and the O(N) costs for cross-connects, the thought of  
> revisiting the old peering coops presented itself again.

The $4/Mbps & under prices is usually reserved for very large CIR or  
full pipe.  With a full pipe, assuming you are very, very, very good,  
you still pay $5 or more.  (I'm assuming a max of 8 Gbps on a 10G  
pipe, which seems overly optimistic IMHO.)  But that's not important  
to the discussion.

> Assuming 10G PNI model: Assuming even nominal cross-connect fees of  
> $100-$300/month per fiber pair, plus router port costs for each  
> private peer (assuming you aren't at >10% utilization on the port)  
> at a commercial exchange, you are eating a pretty significant cost  
> per megabit you are actually moving. (plug in your numbers here).  
> Assumption: Above 1Gb/s utilization, this makes sense or you are  
> counting on growth.

Define "significant"?

Taking 1 Gbps (10%), and assuming even 20K per 10G port over 2 years,  
adding in $300/month for the x-conn, you are still looking at barely  
over $1/Mbps.  If you have more than 10% utilization, that number goes  
down.  Is that significant?  Compared to what?  Transit?

I would say a 75% price reduction is pretty significant.  Plus you  
haven't considered CapEx cost for the transit ports.

> Below 10% you would normally go to a paid peering fabric where you  
> are paying cross connect + a flat port charge + router port for 1->N  
> peers and hoping that enough utilization occurs that you get >10%  
> utilization (to recover capex, opex, etc) and then whatever  
> additional utilization you need to cover the flat port charge or you  
> are counting on growth.

Here we agree.  The port fee even on european IXes is measured in  
1000s of $$ per month.  And don't get me started on US or Japanese  

> A "coop", best-effort switch fabric colo'd at a few sites would  
> allow participants to peer off traffic at a price of the order of a  
> single cross-connect (~$500/month per 10G port is possible, maybe  
> less), private-VLANs all-around, or to only-mutually approved peers  
> (e.g. via an automated web interface, prior art) to avoid many of  
> the /old/ issues. No requirement for multi-lateral peering. You  
> could peer, sell transit, buy transit, multicast, etc.
> The way I figure it, it removes approximately an order of magnitude  
> from the operational cost of peering with more than a handful of  
> your largest single talkers. Especially as 100G LAN Ethernet becomes  
> production before 100G WAN connections become commonplace. Economic  
> theory (assuming that worked on the Internet) suggests this would  
> allow for the increase in number of peers by approximately an order  
> of magnitude (maybe more).

Sorry, I can't get there.

First, the "largest single talkers" would not be on a shared fabric,  
they'd be on dedicated ports, so this idea doesn't help.

For the medium to small guys, I think it's a great idea.  Look at SIX,  
TorIX, PaNAP, etc.  But shaving an _order of magnitude_ off?  No, I  
don't see it.  CapEx alone is more than 10% of your cost.  (Well,  
unless you get Japanese IX ports or the most expensive US IX ports.)

Perhaps I'm lost or confused?  Can someone help me understand?


More information about the NANOG mailing list