Verizon Policy Statement on Net Neutrality

Dave Taht dave.taht at
Sun Mar 1 18:35:50 UTC 2015

I am not normally, willingly, on nanog. My emailbox is full enough. I
am responding, mostly, to a post I saw last night, where the author
complained about the horrid performance he got when attempting a
simultaneous up and download on a X/512k upload DSL link.

That is so totally fixable now, at speeds below 60mbit, with any old
cast off home router that I had to reply...

Honestly I had assumed that everyone here has the chops to fix their
own networks (home and business), in circumstances of high latency
under load caused by bufferbloat - *by now* - and I hadn't spent any
time on this list at all.

A DSL product, running openwrt, made in australia, - was the first DSL
device to get the excess buffering in the DSL driver ripped out and
fq_codel tested. I was over at David Woodhouse's house in England
while he fixed it - which was at LEAST 3 years ago. And he's been
running it with openwrt and those fixes ever since. The name began
with a T, it was a geode, I can't remember the name of it now.

These were the results we got on DSL on an old modem that supported
pause frames:

These are the improvements in bandwidth and latency under load - in
both directions - we commonly get on cable modems, using the
sqm-scripts now in openwrt and working on any linux box you care to

Probably the shortest talk I have ever given on these topics (23
minutes long) was at uknof here, where in particular, I demoed the
improvements in web load time that are now possible. 2 years ago.

See also:

And recently I gave much longer talk, which FINALLY includes some bits
on how we intend to now focus on *vastly improving wifi*, at nznog,
which starts at 2:05 on friday morning here:

I am tired of looking at myself, and I have to say that the talk
before mine was WONDERFUL - the guy went into all sorts of new ways to
find latency events, and filter out any false positives and provide
new kinds of alerts to operators.

And the talk after, from cloudflare, depressing as hell.

I will gladly give another bloat talk to nannogers if that helps at
some future conference, but jeeze, this stuff is so easy to fix now,
and everyone involved is tired of repeating themselves, especially me.
but since I haven't ranted here yet... and only intend to do this
once, here I go....


Since being developed, the core BQL and fq_codel code has become fully
available in every linux distribution I know of. The most advanced
versions of it are in the "sqm-scripts" that are part of the openwrt
"chaos calmer" work - notably - unlike every other shaper I know of,
it can correctly compensate for PPPoe and DSL framing problems, and it
automatically handles the problems that codel has on links below
2.5Mbits. We worked for over a year to get that right - fixed all the
bugs in htb, mainlined and made available for free how to do it all
right, as of Linux 3.10.12 and later, and all that logic is in those
sqm-scripts - which work best on openwrt but also work on any linux
distro with a couple tweaks.

(and since then we have worked to pour it all into C with even simpler
configuratiojn, that work is not done yet, please feel free to come

So anyone here, with a spare 60-90 bucks, 5 minutes, and the right
re-flashable router
no longer has cause to complain about high jitter and latency, even on
the slowest
and most asymmetric links at home, or in their businesses.

Benchmarks of the "fq" portion of fq_codel show it as better than
"sfq", and the codel portion, way better than RED.

It helps of course, to do valid benchmarking of the real problems in
your links - in your switches - and in your routers - at nanog scales
- so we have developed a suite of tests you can use called "rrul" real
time response under load - available for free as part of
"netperf-wrappers" -

The server for which works on everything (netperf is very portable),
and the test client driver, analysis tools and gui, work on any linux
system and can be made to work on OSX via macports (I was unsuccessful
at brew). The data it collects is your own, and aggregatable, and you
don't need to share it with anyone if you don't want to.

I would certainly like it, if after evaluating and then fixing bits of
your own network that way, that anyone doing so, would volunteer to go
fix two other networks, and get the people running those networks to
go fix two other networks, each, and so on. And of course, feel free
to nag and publicly embarrass those providing busted, bloated CPE,
DSLAMS, BRAMS, and CMTSes, etc to actually deploy this stuff on their
side, so we all can move on, and all have way better networks.

Anyone here *making linux based CPE*, can merely update their code to
something modern, containing these algorithms.  Backports are
possible, but so far, I have been quite depressed by the overall buggy
results I have seen from the home router makers attempting to so. All
the streamboost products I have tested are defective in some respect
or another - so I am back to recommending people just reflash their
firmware ("friends don't let friends run factory firmware") with
something readily available and good off of openwrt's unbelievable
large list.

I have a fondness for the ar71xx products as those generally have the
best drivers (ath9k) for wifi.

I know full well, that Linux does not rule the world (yet), and most
of my ire these days is aimed at those that are not paying attention
or helping fix their crappy systems.

While half of the solution - codel - is already available as patches
to at least one BSD version... I am still waiting patiently for
someone to code up a version of fq_codel for BSD. fq_codel was written
in a single saturday afternoon, and mainlined in the next week - and
despite trying, we have been unable to come up with anything more than
marginally better. I have been reluctant to do that port personally as
transliterating from GPL to BSD is something that someone else should
do, based on the IETF internet draft we produced.

We (on the codel or bloat email lists) will gladly review patches and
performance data, however.

If you are stuck with older gear that can't be updated, perhaps you
can apply HFSC + SFQ which works pretty well at lower bandwidths, but
has problems that I documented here:

As for fixing your interconnects - or any other places you have *any*
bufferbloat and/or bad latency - the fq_codel algorithm is incredibly
light-weight, and works at 10GE and above. (software rate shaping, not
so much)

Netperf-wrapper also scales to 10GigE. In fact, it is being used to
test stuff running at 40GigE. Most other tools for testing for
bandwidth and latency, start showing erroneous results above 20Mbit.

I too would like asymmetric networks to behave better, but these are
the best tools we got to fix what we have. Go forth, test, and deploy!

A plea: please stop debating about policy and just deploy s**t that works.

[1]  Quite a few bits of DSL firmware are a problem. The only truly
"perfect" DSL firmware I know of is in's revolution v6 router
- which was deployed august, 2012 - with a DRR + fq_codel based
implementation that works *really well* at line rate, whatever it is -
no software shaping required.

On Sun, Mar 1, 2015 at 3:40 AM, Aled Morris <aledm at> wrote:
> On 1 March 2015 at 03:41, Barry Shein <bzs at> wrote:
>> Previously all residential service (e.g., dial-up, ISDN) was
>> symmetrical.
> The rot set in with V.90 "56k" modems - they were asymmetric - only the
> downstream was 56k.  The only way to achieve this in the analogue realm was
> by digital synthesis at the head-end, i.e. the T1/E1 handoff to the ISP.
> The upstream from the subscriber didn't have a clean interface so was still
> using 33.6k.
> Sadly we don't have many "killer applications" for symmetric residential
> bandwidth, but that's likely because we don't have the infrastructure to
> incubate these applications.
> It's a chicken and egg situation - of course the average consumer today
> will say they "don't need" symmetric, but you could have asked them twenty
> years ago and they'd have said they didn't need the Internet at all.  Or
> smartphones.

I agree totally with this sentiment - if we had symmetric edge
networks, we would have come up with ways to use them.

offsite backup would be a lot more common, in particular.

> This all suits the telcos and cablecos very nicely - they are happy when
> their customers are passive consumers of paid content and services.  It
> gives them control.
> I don't think it's a conspiracy, but it suits the big players not to "fix"
> the "problem" since they don't perceive it as being one.
> Aled

Dave Täht
Let's make wifi fast, less jittery and reliable again!

More information about the NANOG mailing list