free collaborative tools for low BW and losy connections

Joe Greco jgreco at ns.sol.net
Sun Mar 29 22:14:07 UTC 2020


On Sun, Mar 29, 2020 at 10:31:50PM +0100, Nick Hilliard wrote:
> Joe Greco wrote on 29/03/2020 21:46:
> >On Sun, Mar 29, 2020 at 07:46:28PM +0100, Nick Hilliard wrote:
> >>>That's so hideously wrong.  It's like claiming web forums don't
> >>>work because IP packet delivery isn't reliable.
> >>
> >>Really, it's nothing like that.
> >
> >Sure it is.  At a certain point you can get web forums to stop working
> >by DDoS.  You can't guarantee reliable interaction with a web site if
> >that happens.
> 
> this is failure caused by external agency, not failure caused by 
> inherent protocol limitations.

Yet we're discussing "low BW and losy(sic) connections".  Which would be
failure of IP to be magically available with zero packet loss and at high
speeds.  There are lots of people for whom low speed DSL, dialup, WISP,
4G, GPRS, satellite, or actually nothing at all are available as the 
Internet options.

> >>>Usenet message delivery at higher levels works just fine, except that
> >>>on the public backbone, it is generally implemented as "best effort"
> >>>rather than a concerted effort to deliver reliably.
> >>
> >>If you can explain the bit of the protocol that guarantees that all
> >>nodes have received all postings, then let's discuss it.
> >
> >There isn't, just like there isn't a bit of the protocol that guarantees
> >that an IP packet is received by its intended recipient.  No magic.
> 
> tcp vs udp.

IP vs ... what exactly?

> >>Flood often works fine until you attempt to scale it.  Then it breaks,
> >>just like Bj??rn admitted. Flooding is inherently problematic at scale.
> >
> >For... what, exactly?  General Usenet?
> 
> yes, this is what we're talking about.  It couldn't scale to general 
> usenet levels.

The scale issue wasn't flooding, it was bandwidth and storage.  It's 
actually not problematic to do history lookups (the key mechanism in 
what you're calling "flooding") because even at a hundred thousand per 
second, that's well within the speed of CPU and RAM.  Oh, well, yes, 
if you're trying to do it on HDD, that won't work anymore, and quite 
possibly SSD will reach limits.  But that's a design issue, not a scale
problem.

Most of Usenet's so-called "scale" problems had to do with disk I/O and
network speeds, not flood fill.

> >Perhaps, but mainly because you
> >do not have a mutual agreement on traffic levels and a bunch of other
> >factors.  Flooding works just fine within private hierarchies and since
> >I thought this was a discussion of "free collaborative tools" rather than
> >"random newbie trying to masochistically keep up with a full backbone
> >Usenet feed", it definitely should work fine for a private hierarchy and
> >collaborative use.
> 
> Then we're in violent agreement on this point.  Great!

Okay, fine, but it's kinda the same thing as "last week some noob got a
1990's era book on setting up a webhost, bought a T1, and was flummoxed
at why his service sucked."

The Usenet "backbone" with binaries isn't going to be viable without a
real large capex investment and significant ongoing opex.  This isn't a
failure in the technology.

> >>delivered it.  TAKETHIS managed to sweep these problems under the
> >>carpet, but it's a horrible, awful protocol hack.
> >
> >It's basically cheap pipelining.
> 
> no, TAKETHIS is unrestrained flooding, not cheap pipelining.

It is definitely not unrestrained.  Sorry, been there inside the code.
There's a limited window out of necessity, because you get interesting
behaviours if a peer is held off too long.

> >If you want to call pipelining in
> >general a horrible, awful protocol hack, then that's probably got
> >some validity.
> 
> you could characterise pipelining as a necessary reaction to the fact 
> that the speed of light is so damned slow.

Sure.

> >>which is mostly because there are so few large binary sites these days,
> >>i.e. limited distribution model.
> >
> >No, there are so few large binary sites these days because of consolidation
> >and buyouts.
> 
> 20 years ago, lots of places hosted binaries.  They stopped because it 
> was pointless and wasteful, not because of consolidation.

I thought they stopped it because some of us offered them a better model 
that reduced their expenses and eliminated the need to have someone who was 
an expert in an esoteric '80's era service, while also investing in all the
capex/opex. 

Lots of companies sold wholesale Usenet, usually just by offering access to 
a remote service.  As the amount of Usenet content exploded, the increasing 
cost of storage for a feature a declining number of users was using didn't 
make sense. 

One of my companies specialized in shipping dreader boxes to ISP's and 
letting them backend off remote spools, usually for a fairly modest cost 
(high three, low four figures?).  This let them have control over the service
that was unlike anything other service providers were doing.  Custom groups,
custom integration with their auth/billing, etc., required about a megabit
of bandwidth to maintain the header database, anything else was user traffic.
Immense flexibility without all the expense.

It was totally possible to keep things going.  The reason text Usenet tended 
to lose popularity has more to do with it representing a technology that had
difficulty evolving.  As HTTP grew more powerful, easy signups and user avatars
were attractive things, and meant competing forumwares developed powerful
features that Usenet could never really have.

Consolidation and buyouts is something that happened more in the last ten
years.  Sorry for any confusion there.

I spent enough years on this stuff that it matters to me that we're honest
about these things.  I don't mind that Usenet is a declining technology and
that people don't use it because it is difficult in comparison to a webforum.

Usenet is a great technology for doing collaboration on low bandwidth and 
lossy connections.  It will get the traffic through in situations where
packet loss would make interactive traffic slow-to-impossible.  I don't
expect to see any great upswing in Usenet development at this point, though.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"The strain of anti-intellectualism has been a constant thread winding its way
through our political and cultural life, nurtured by the false notion that
democracy means that 'my ignorance is just as good as your knowledge.'"-Asimov



More information about the NANOG mailing list