Network end users to pull down 2 gigabytes a day, continuously?

Sun Jan 21 12:14:56 UTC 2007

 Said Sprunk:

Caching per se doesn't apply to P2P networks, since they already do that

> as part of their normal operation.  The key is getting users to contact
> peers who are topologically closer, limiting the bits * distance
> product.  It's ridiculous that I often get better transfer rates with
> peers in Europe than with ones a few miles away.  The key to making
> things more efficient is not to limit the bandwidth to/from the customer
> premise, but limit it leaving the POP and between ISPs.  If I can
> transfer at 100kB/s from my neighbors but only 10kB/s from another
> continent, my opportunistic client will naturally do what my ISP wants
> as a side effect.
>
> The second step, after you've relocated the rate limiting points, is for
> ISPs to add their own peers in each POP.  Edge devices would passively
> detect when more than N customers have accessed the same torrent, and
> they'd signal the ISP's peer to add them to its list.  That peer would
> then download the content, and those N customers would get it from the
> ISP's peer.  Creative use of rate limits and acess control could make it
> even more efficient, but they're not strictly necessary.

Good thinking. Where do I sign? Regarding your first point, it's really
surprising that existing P2P applications don't include topology awareness.
After all, the underlying TCP already has mechanisms to perceive the
relative nearness of a network entity - counting hops or round-trip latency.
Imagine a BT-like client that searches for available torrents, and records
the round-trip time to each host it contacts. These it places in a lookup
table and picks the fastest responders to initiate the data transfer. Those
are likely to be the closest, if not in distance then topologically, and the
ones with the most bandwidth. Further, imagine that it caches the search -
so when you next seek a file, it checks for it first on the hosts nearest to
it in its "routing table", stepping down progressively if it's not there.
It's a form of local-pref.

The third step is for content producers to directly add their torrents
> to the ISP peers before releasing the torrent directly to the public.
> This gets "official" content pre-positioned for efficient distribution,
> making it perform better (from a user's perspective) than pirated
> content.
>
> The two great things about this are (a) it doesn't require _any_ changes
> to existing clients or protocols since it exploits existing behavior,
> and (b) it doesn't need to cover 100% of the content or be 100%
> reliable, since if a local peer isn't found with the torrent, the
> clients will fall back to their existing behavior (albeit with lower
> performance).

Importantly, this option makes it perform better without making everyone
else's perform worse, a big difference to a lot of proposed QOS schemes.
This non-evilness is much to be preferred. Further, it also makes use of the
Zipf behaviour discussed upthread - if 20 per cent of the content and 20 per
cent of the users eat 80 per cent of the bandwidth, forward-deploying that
20 per cent of the content will save 80 per cent of the inter-provider
bandwidth (which is what we care about, right, 'cos we're paying for it).

> One thing that _does_ potentially break existing clients is forcing all
> of the tracker (including DHT) requests through an ISP server.  The ISP
> could then collect torrent popularity data in one place, but more
> importantly it could (a) forward the request upstream, replacing the IP
> with its own peer, and (b) only inform clients of other peers (including
> the ISP one) using the same intercept point.  This looks a lot more like
> a traditional transparent cache, with the attendant reliability and
> capacity concerns, but I wouldn't be surprised if this were the first
> mechanism to make it to market.

It's a nice idea to collect popularity data at the ISP level, because the
decision on what to load into the local torrent servers could be automated.
Once torrent X reaches a certain trigger level of popularity, the local
server grabs it and begins serving, and the local-pref function on the
clients finds it. Meanwhile, we drink coffee. However, it's a potential DOS
magnet - after all, P2P is really a botnet with a badge. And the point of a
topology-aware P2P client is that it seeks the nearest host, so if you
constrain it to the ISP local server only, you're losing part of the point
of P2P for no great saving in peering/transit.

However, it's going to be competing with a deeply-entrenched pirate
> culture, so the key will be attractive new users who aren't technical
> enough to use the existing tools via an easy-to-use interface.  Not
> surprisingly, the same folks are working on deals to integrate BT (the
> protocol) into STBs, routers, etc. so that users won't even know what's
> going on beneath the surface -- they'll just see a TiVo-like interface
> and pay a monthly fee like with cable.

As long as they don't interfere with the user's right to choose someone
else's content, fine.

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20070121/34c1266d/attachment.html>