Internet Backbone Index

Sat Jun 28 12:05:42 UTC 1997

Sean,

Do you have a pointer to the raw data?  I couldn't find it
on the site.

-scott

At 04:25 PM 6/27/97 -0500, Sean Donelan wrote:
>Ok, so I said I wasn't going to comment on the methodology, I lied.
>I wouldn't say the Keynote study is the worst ever.  There are some
>really rotten studies in the fields of psychology and sociology.
>
>Since the Boardwatch/Keynote study didn't 'test' DRA Net, I guess I'm
>one of the few independent, disinterested parties to comment on the
>study's methods.
>
>A problem with the Keynote study is it seems very dependent on the
>location, type and connections of the testing platforms.  Keynote
>mentions that connections from Dallas and Phoenix were slow to 'every'
>backbone site.  This would indicate some systematic problem with
>the testing sites.  Perhaps the results are even more dependent on
>the testing systems than the systems under test.  There are also
>problems with outlier data points.  For example, elsewhere on the
>Keynote site, the MCI web site had very fast access from 28 test
>sites (< 4secs), and very slow access from one test site in
>philly (> 14secs).  Mixing and matching data points, if you left
>out the one outlier data point, MCI would have been faster than
>Savvis.  So I don't know if the rankings are very meaningful if a
>single test site can have such a pronounced effect.
>
>Unlike a scientific study, there doesn't seem to be enough information
>to independently reproduce the results, so I'm just going from the bits
>and pieces I can glean from the Keynote pages, press release, and article.
>
>>Concerning Internet performance, there have always been a variety of ways
>>of measuring it. It all depends on what you are really trying to measure.
>>The Keynote study is attempting to measure something to which the average 
>>Internet user (not engineers) can relate.  However, There are also clearly 
>>the possibility of artifacts in the data because of the testing machine's 
>>TCP stack or other issues (Vern Paxson has covered these issues at NANOG 
>>and IETF meetings over the last few years). Checking their web site, their 
>>software appears to run on top of the TCP stacks of many systems, so the 
>>known artifacts of some of these platforms could be an issue.
>-- 
>Sean Donelan, Data Research Associates, Inc, St. Louis, MO
>  Affiliation given for identification not representation
>
>