Network end users to pull down 2 gigabytes a day, continuously?

Sun Jan 21 03:19:38 UTC 2007

Thus spake "Dave Israel" <davei at otd.com>
> The past solution to repetitive requests for the same content has been
> caching, either reactive (webcaching) or proactive (Akamaizing.)  I
> think it is the latter we will see; service providers will push
> reasonably cheap servers close to the edge where they aren't too
> oversubscribed, and stuff their content there.  A cluster of servers
> with terabytes of disk at a regional POP will cost a lot less than
> upgrading the upstream links.  And even if the SPs do not want to
> invest in developing this product platform for themselves, the price
> will likely be paid by the content providers who need performance to
> keep subscribers.

Caching per se doesn't apply to P2P networks, since they already do that 
as part of their normal operation.  The key is getting users to contact 
peers who are topologically closer, limiting the bits * distance 
product.  It's ridiculous that I often get better transfer rates with 
peers in Europe than with ones a few miles away.  The key to making 
things more efficient is not to limit the bandwidth to/from the customer 
premise, but limit it leaving the POP and between ISPs.  If I can 
transfer at 100kB/s from my neighbors but only 10kB/s from another 
continent, my opportunistic client will naturally do what my ISP wants 
as a side effect.

The second step, after you've relocated the rate limiting points, is for 
ISPs to add their own peers in each POP.  Edge devices would passively 
detect when more than N customers have accessed the same torrent, and 
they'd signal the ISP's peer to add them to its list.  That peer would 
then download the content, and those N customers would get it from the 
ISP's peer.  Creative use of rate limits and acess control could make it 
even more efficient, but they're not strictly necessary.

The third step is for content producers to directly add their torrents 
to the ISP peers before releasing the torrent directly to the public. 
This gets "official" content pre-positioned for efficient distribution, 
making it perform better (from a user's perspective) than pirated 
content.

The two great things about this are (a) it doesn't require _any_ changes 
to existing clients or protocols since it exploits existing behavior, 
and (b) it doesn't need to cover 100% of the content or be 100% 
reliable, since if a local peer isn't found with the torrent, the 
clients will fall back to their existing behavior (albeit with lower 
performance).

One thing that _does_ potentially break existing clients is forcing all 
of the tracker (including DHT) requests through an ISP server.  The ISP 
could then collect torrent popularity data in one place, but more 
importantly it could (a) forward the request upstream, replacing the IP 
with its own peer, and (b) only inform clients of other peers (including 
the ISP one) using the same intercept point.  This looks a lot more like 
a traditional transparent cache, with the attendant reliability and 
capacity concerns, but I wouldn't be surprised if this were the first 
mechanism to make it to market.

> I think the biggest stumbling block isn't technical.  It is a question
> of getting enough content to attract viewers, or alternately, getting
> enough viewers to attract content.  Plus, you're going to a format
> where the ability to fast-forward commercials is a fact, not a risk,
> and you'll have to find a way to get advertisers' products in front of
> the viewer to move past pay-per-view.  It's all economics and politics
> now.

I think BitTorrent Inc's recent move is the wave of the short-term 
future: distribute files freely (and at low cost) via P2P, but 
DRM-protect the files so that people have to acquire a license to open 
the files.  I can see a variety of subscription models that could pay 
for content effectively under that scheme.

However, it's going to be competing with a deeply-entrenched pirate 
culture, so the key will be attractive new users who aren't technical 
enough to use the existing tools via an easy-to-use interface.  Not 
surprisingly, the same folks are working on deals to integrate BT (the 
protocol) into STBs, routers, etc. so that users won't even know what's 
going on beneath the surface -- they'll just see a TiVo-like interface 
and pay a monthly fee like with cable.

S

Stephen Sprunk         "God does not play dice."  --Albert Einstein
CCIE #3723         "God is an inveterate gambler, and He throws the
K5SSS        dice at every possible opportunity." --Stephen Hawking 