Reducing Usenet Bandwidth

Sun Feb 17 23:11:23 UTC 2002

At 11:57 PM 2/17/2002 +0100, Iljitsch van Beijnum wrote:
>On Sun, 17 Feb 2002, Eliot Lear wrote:
>
> > This is the art of content delivery and caching.
>
>Actually _delivery_ is only part of the problem: it assumes the content is
>available, people know enough about it to be able to decide they want it,
>and they know where it is and how to request it. Obviously, delivery is an
>important aspect of the whole process to optimize, since it takes a lot of
>bandwidth, depending on the type of content. But distributing the
>meta-information is even harder, and potentially more expensive.

The only expensive part about dealing with the meta-information is adapting 
existing technology to point at URLs.  Passing the meta-information is 
several orders of magnitude less expensive than the actual files 
themselves.  Managing the meta-data (i.e., CPU/memory) is negligable 
compared to the cost and latency of retrieving that data.  The retrieval of 
XOVER information and such scales well.  That's been the lesson of the last 
few years.  And content delivery networking mechanisms can be used for it.

>The
>failure of Usenet to effectively do it demonstrates this: because
>selection is pretty much impossible, you have to deliver everything to a
>place very near the potential user, even the stuff that is of no interest
>to any user.

To call USENET a failure is a bit of a stretch.  But its scaling point is 
on the side of the reader.  That is solved today through stronger search 
capabilities of various search engines that were merely a gleam in several 
people's eye, even as late as 1990.  It wasn't until WAIS came about that 
USENET became more searchable, and then the economics began to shift.  Not 
that people didn't try to do selection.  Brad Templeton had a 
semi-automated feedback mechanism with Clarinet that he gave away to his 
customers.

>In reality, people don't want to think about it. How much am I willing to
>pay for the privilige of posting this message to NANOG? And you to read
>it? If we both apply the hourly rate we bill our customers to the time we
>spend on it (because we could have been doing work that actually pays
>money instead), probably more than we realize. On the other hand, If I had
>to cough up some money right here, right now to post this, I probably
>wouldn't.

Nobody wants to think about it until real money is involved.  And while 
text discussions don't involve a whole lot of real money, this conversation 
was started by people who are tired of transporting warez and having to pay 
for it.

Eliot