Vancouver IXP - VanTX - BCNet

Wed Aug 21 19:02:44 UTC 2013

Omnibus reply warning.  Skip this one unless you're really into IXP trivia.

On Aug 21, 2013, at 7:52 AM, Randy Bush <randy at psg.com> wrote:
>> - New participation is not too rigorously constrained (at least a
>>  domestic ISP new market entrant should be able to participate) 
> 
> imiho, it is also nice if non-isp folk can participate, content, etc.

Of course, and that is best-practice.  I was listing the things that were, to my mind, bare minimums.  I think we can agree that if two ISPs interconnect with each other, but prohibit a third or subsequent ISPs from interconnecting, that does not constitute an IXP.  There are a small but significant subset of IXPs are the only option within their regions and are universally recognized to be IXPs, yet still place some restrictions upon who can participate, often requiring that participants hold a national ISP license.  We generally work to influence them to ease or remove those restrictions over time.  For example, we've succeeded in getting that restriction lifted from the Beirut exchange, while we're still working to move the management of the Buenos Aires exchange toward best-practices in this regard.

>> - Participants do not receive a metered-rate bill based on utilization
> 
> i am not sure i understand why.  just seems a finer
> grained case of 100mb for $1, 1g for $5, and 10g for $20 or whatever.

Certainly that's one way of looking at it.  Can you think of any examples of something you'd consider to unequivocally be an IXP, yet charges a participant 1% more if they use 50.5mbps than if they use 50mbps?  Generally this is one of the hallmarks of a layer-1/2 carrier that's trying to pass itself off as an IXP for marketing purposes.  The principle drawback of such a charging scheme is that it removes the efficiency-of-scale incentive for ISPs to use the IXP more, and therefore removes their ability to charge their customers less as they use the IXP more, and thus the "exchange" fails to grow.  Hypothetically, if the price were low enough, none of this would be a significant factor or disincentive, but in twenty years of IXPs and things-that-tried-to-be-IXPs, I haven't seen a successful example of such, while seeing quite a few that failed, where this was the primary distinguishing feature.  It's not a common thing, but it's been a big red flag when it has shown up.

> i would add carrier neutrality, i can haul fiber from anyone into
> the exchange.  this is pretty critical in the exchanges where i have
> played.

I'd say that carrier neutrality is a hard requirement in markets where that's a decision being made by the IXP (or more often the colo that hosts the IXP).  There are a lot of markets that don't have competitive carriers, so this issue isn't yet tested at the time we're trying to figure out what's what.  For instance, It was twelve years between the point at which the first IXP was formed in Singapore, and the first time anyone was able to run a competitive piece of fiber into it.

>> Peer1 exchanges are only open to Peer1 customers, I believe. At least,
>> that's how it worked in Toronto the last time I looked.
> 
> that is not an exchange.  most isps have switches in their transit
> infrastructure.

I agree that ISPs have switches, and will usually happily sell transit (or colo) to customers connected to those switches, and that that doesn't constitute an exchange.  Peer1 presents themselves as a colocation provider (that also sells hosting). I think most people other than Peer1 would agree that Peer1's facilities don't constitute exchanges, and they're not marked as exchanges by our staff, in the directory.  However, the line between Peer1 and, for instance, Equinix or Global Switch, is fairly fine…  I think you'd find general consensus that Equinix switches are IXPs, and Equinix is a colocation provider (that also occasionally sells transit, to some of their customers, on occasion).  So while the position you're expressing is certainly the majority position, it's something of a matter of degree and focus, and to some degree a judgment in the eye of the beholder, rather than something easily expressible in a simple objective rule.

On Aug 21, 2013, at 7:38 AM, Clayton Zekelman <clayton at MNSi.Net> wrote:
> Just wondering aloud if an ISP that did have commercial interest could run a non-member driven exchange point successfully as long as they had pricing and policies that were similar to member driven exchange points.

ISPs tend to be very pragmatic on the issue of facility neutrality, whereas folks like regulators and those who are selling a product in the area get more religious about it.  So ISPs don't tend to care much whether a facility is neutral, or run by one of their competitors, if they can see a clear value proposition in using it.

That said, the price-point of the non-neutral for-profit IXP you postulate is not necessarily equal to or greater than zero…  First of all, operating a switch is well within the core competency of ISPs, so they all know that the cost of doing it themselves is de minimis. Second, they know that the capex of building out to someone else's facility can be substantial, so while they're willing to do that when the future value of that investment, and their ability to recoup it, is protected, they generally won't do so if they can't see very clear protections (cash and contracts, not assertions and good-will).  Third, they know that, while they have to make that infrastructural investment to reach the exchange, the owner of the facility does not.  When the owner is not an ISP, that's not an issue, but when the owner is an ISP, and one of their competitors, it constitutes a significant relative competitive disadvantage.  Although both parties lose in an absolute sense if they fail to exchange traffic, the other guy didn't have to drop a lot of money, and doesn't have as much to lose, so can afford to play hardball in negotiating over who gets to keep excess rent.

In practice, you don't see this arrangement much (non-neutral IXPs) for two reasons: when IXPs are established through an open stakeholder process, all stakeholders make clear that they'd be happy to host the exchange within their facility, and once they've all gotten that off their chests, they move on to deciding upon a jointly-acceptable neutral location; and when a for-profit entity unilaterally establishes a non-neutral exchange, ISPs generally don't choose to use it, and it fails through the action of market forces.

The halfway-case, neutral for-profit exchanges, generally a feature of neutral for-profit colocation facilities, are an interesting compromise and are often quite successful in the marketplace.  So it's useful to look at them as a point of comparison.  ISPs still know that they can do the switch thing themselves just as easily, so the cost still needs to be relatively low, and the points of value need to be things that are more difficult for an ISP to do internally.  First and foremost, the neutrality and the large datacenter facility attract other people to talk to. Second, also very compelling, getting to take advantage of the benefits of scale; generators, security, etc., in a location where you only need one or two cabinets.  Third, many of these businesses make their money elsewhere, through real-estate speculation or by issuing shares, so the services ISPs are buying are subsidized by those other lines of business, and are, literally, less expensive than if the ISP were to build it themselves.

> Community of interest of course is the other magical ingredient that is necessary.  Not sure how many ISPs would want to peer in Windsor...

> If there was a legitimate community of interest for establishing an IXP here, we could do it, but alas, as has been pointed out, the case for TorIX is so compelling, and so much needs to flow through Toronto regardless, it seems the natural place to interconnect.

Folks in London (England, not Ontario) said the same thing about Washington D.C. before they got an IXP of their own up, as well. It's very easy to look at the status-quo and observe that it's functioning, while it's not always so easy to imagine how things will be better, if things pan out, in five or ten years.  My experience leads me to believe that Windsor can support an IXP, and that if you don't over-spend and gold-plate it, it'll make money for you, and will grow over time.  As will TorIX.  It's quite likely that an exchange in Windsor will never be as large as one in Toronto, but that's no reason not to do it.  A restaurant or gas station in Toronto might be more successful than one in Windsor, but it doesn't mean that nobody bothers to start a restaurant or gas station in Windsor.  Same thing.  Speed times distance equals cost, and always will.  Exchange traffic more locally to keep speeds high and costs low, and stay competitive.  If you just use an IXP in Toronto, ISPs in Toronto will always be more competitive than you will, because their cost-of-goods will always be lower.

On Aug 21, 2013, at 10:27 AM, "William F. Maton Sotomayor" <wmaton at ottix.net> wrote:
> While it is admirable that CIRA (and probably other similar counterparts are watching) looking to establish IXPs…

I think CIRA has been very clear that they're not trying to establish IXPs, they're trying to provide any desired support to locally-based IXP efforts.  There's a big difference.  In order for an IXP to succeed, it needs a constituency of ISPs who understand that it's critical to their financial success.  You can't just drop in and build an IXP for someone, and expect it to still be there a year later.

> My anxiety lies with the future:  Given everything that's already been written, are any of these IXPs capable of becoming self-sustaining in the future?  It's a rhetorical question applicable to any starting IXP.

Indeed.  I think that ISPs who understand their business model well enough to understand the effect the IXP will have on their average-per-bit-delivery-cost is essential.  I think it's also essential that they have some basic familiarity with the different ways IXPs can fail, or fail to thrive, so that they can avoid mistakes others have made in the past.  Over-spending, particularly on switches, is a huge killer of IXPs.  Under-provisioning of circuits to the IXP is another big mistake.  Failure to encourage local content and hosting is another.

                                -Bill

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20130821/72f6e438/attachment.sig>