Speedtest site accuracy [was: Bandwidth issues in the Sprint network]

Wed Apr 9 16:02:42 UTC 2008

Jeff Shultz wrote:
>  Regarding speed test software, what are people running, and what do you 
> think of it?

That's a good question and one I was going to ask myself if no one else did.

We're currently using the AuditMyPC speedtest:

http://www.auditmypc.com/broadband-speed-test.asp

I'd like to using the same speedtest app as speedtest.speakeasy.net:

http://www.ookla.com/

The AuditMyPC one works ok most of the time.  However it freaks out when 
you introduce packet loss.  I've been impressed with the ookla speedtest 
app.  I just haven't convinced the right people to buy a copy.

As with any flash or java based speedtest the speeds of the PC affect 
the outcome just as much as the speed of their connection.  For example 
I can do speedtests from my old laptop (2G P4 w/ 2GB RAM running XP SP2 
& 1.5yrs of crud) and the same test with my new laptop (dual-core 2.2G 
P4 w/ 4GB RAM running XP SP2 fresh from the box), both laptops connected 
directly to our SP backbone.  My old laptop will get 12/5 on a bad day 
(haven't restarted Windows or FireFox in a few days) whereas my new 
laptop will pull down 30/20 during out peak times without breaking a 
sweat.

Only a minority of customers will have decent PCs at home.  Most 
probably bought the electronics store special (or Wal-Mart).  It has 
just enough RAM to run Windows.  It has the slowest hard drive possible 
to save $5 in manufacturing costs.  It has a bargain basement processor 
with little to no cache.  It's been on the Internet probably without a 
firewall (so who know how many times it's been compromised).  It's been 
used by users who don't have any expertise in security or Internet 
safety so they've probably installed WebShots and WeatherBug and every 
other useless piece of junk out there (Google Desktop), not to mention 
the drive-by installs made possible by IE.  It also came loaded down 
with a bunch of OEM BS that the user will never use.  So we have users 
with low-end PCs, made nearly worthless by the crap that's on them, 
trying run Flash or Java speedtest apps.  Oh, and I forgot to say that 
the speedtest site is one the user found out there on the Internet, 
probably in another country.  They've connected their FiOS equipment to 
an 8-year old LinkSys that tops out at 4Mbps and which also lowers the 
MTU to 1492 just in case it's connected to a DSL modem doing PPPoX. 
Would any technical person really be surprised to hear that the user 
didn't get the full speeds they're paying for?  But it's still the 
provider's fault.

The OS and application crud issues could be mitigated by giving the 
users a boot disk that gives them a basic web browser set to the SP's 
speedtest server.  That would be very hard to do in fact.  There are 
plenty of CD-based Linux distros out there.  You still can't easily get 
around the PC's hardware issues or their network problems created by the 
users themselves.  I don't have a silver bullet for this problem.

We have a local speedtest server for our users.  It's performance 
varies.  It's upload speeds are horrendous.  The download is fairly 
accurate.  I would love to improve this, especially as we roll out our 
4/7/12 broadband packages and also begin out FTTx deployment.

What I'd like to see is major upstreams deploy iperf servers so that we 
smaller upstreams can verify our own SP circuit speeds.  More than once 
I've caught an upstream who couldn't deliver what we were paying for. 
iperf is about as accurate as it gets.  Being able to run a lengthy 
speedtest is very helpful.  Trying to put some stress on a 100Mbps link 
to an upstream with a typical b-band speedtest just doesn't cut it.

Justin