Keynote/Boardwatch Internet Backbone Index A better test!!!

Sun Jun 29 15:38:40 UTC 1997

Jack,

Your study proves that if a user connects to the network, he can
expect better response time surfing www.uu.net than he could expect
from surfing www.mci.com.  While this is interesting, this has little 
or nothing to do with the network performance of either uu.net or
mci.net.  If your goal is to report on the performance that a dedicated
user might expect from uu.net or mci.net, become a customer of either
network and test the response time to the top 100 web sites in the world.
Better yet, use your publication to print measurement software that
your readers might use to test each network and report the results to
you.  

If I had published the article that you just published, I don't know if
I could backpeddle fast enough.  Your method is flawed and your results
are misleading.  

Jeff Young
young at mci.net

> Return-Path: owner-nanog at merit.edu 
> Received: from merit.edu (merit.edu [198.108.1.42])
> 	by postoffice.Reston.mci.net (8.8.5/8.8.5) with ESMTP id QAA25593;
> 	Fri, 27 Jun 1997 16:53:58 -0400 (EDT)
> Received: from localhost (daemon at localhost)
> 	by merit.edu (8.8.5/8.8.5) with SMTP id QAA26583;
> 	Fri, 27 Jun 1997 16:35:35 -0400 (EDT)
> Received: by merit.edu (bulk_mailer v1.5); Fri, 27 Jun 1997 16:34:59 -0400
> Received: (from majordom at localhost)
> 	by merit.edu (8.8.5/8.8.5) id QAA26533
> 	for nanog-outgoing; Fri, 27 Jun 1997 16:34:59 -0400 (EDT)
> Received: from boardwatch.com (ipad1.boardwatch.com [204.144.169.1])
> 	by merit.edu (8.8.5/8.8.5) with ESMTP id QAA26529
> 	for <nanog at merit.edu>; Fri, 27 Jun 1997 16:34:54 -0400 (EDT)
> Received: from ws38.boardwatch.com ([199.33.229.38]) by boardwatch.com
> 	with ESMTP (IPAD 1.52) id 2051600 ; Fri, 27 Jun 1997 14:35:56 EST
> From: "Jack Rickard" <jack.rickard at boardwatch.com>
> To: "Craig A. Huegen" <c-huegen at quadrunner.com>, <nanog at merit.edu>,
>         <marketing at keynote.com>
> Subject: Re: Keynote/Boardwatch Internet Backbone Index  A better test!!!
> Date: Fri, 27 Jun 1997 12:38:02 -0600
> X-MSMail-Priority: Normal
> X-Priority: 3
> X-Mailer: Microsoft Internet Mail 4.70.1155
> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> Message-Id: <199706271835.2051600 at boardwatch.com>
> Sender: owner-nanog at merit.edu
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Length: 4887
> 
> This assumes that you consider web server location and web server
> performance to NOT be a part of overall network performance.  Our view
> steps back a bit from that.  The majority of traffic would appear to be
> webcentric.  From an end users perspective, what does a web site on a
> specific network look like and how does that compare to a web site on
> another network?  There are ENDLESS variables contributing to that
> including intercity links, hub architecture, host hardware, host software,
> peering, connectivity points with other networks, transit agreements, type
> of routers, ATM switching (or not).  All contribute.  We think most people
> notice Internet performance (or lack thereof) while viewing world wide web
> pages.  If we measure such page transits, the results are indicative of the
> accumulation of ALL of those factors.  The web sites chosen were on the
> network under study, operated by the network, and under their control. 
> They have total control over the hardware and software used, how it is
> connected to their network, just as they have control over all other
> aspects of their networks.  Does web server performance affect the results?
>  I would hope so.   Can we break it down into what is purely web server
> hardware performance, what is web server software performance, what is NIC
> card on the web server, what is the impact of the first router the web
> server is connected to, what is the impact of hub design and the interface
> between IP routing and ATM switching, what part is the impact of
> interconnections with other networks, what part is peering, what part is
> just goofy router games?  Uh,,, NO we can't.  
> 
> I would posit that it is only the network engineers at the heart of this
> that would or should care.  I don't know at this point what portion of the
> equation can be levied on web servers and I don't think anyone else can
> either.  I have held for several years that the performance breakdown is in
> the "last inch" of the Internet between the drive controller and the disk
> surface.  But in working with Keynote, I generated the broad theory that if
> that view held true, then by massively averaging measurements across time
> and geography, we should flatline the Internet.  In other words, all
> results should factor to zero relatively.  They didn't.  They didn't to a
> shocking degree.  And at this point I am under the broad assumption that
> server performance doesn't account for all of it, perhaps little of it. 
> But I could be widely wrong on the entire initial assumption.
> 
> In any event, the networks have total control and responsibility for their
> own web servers, much as they do for their own network if you define that
> as something separate from their networks.  We measured web page downloads
> from an end user perspective, and those are the results in aggregate.  If
> it leads to a flurry of web server upgrades, and that moves the numbers,
> we'll know more than we did.  If it leads to a flurry of web server
> upgrades, and it FAILS to move the numbers, that will tell us something as
> well.  
> 
> Our broad theory is that nothing is going to improve as long as anything
> you do doesn't count and is not detectable by anyone anywhere.  If a
> particular network can move their results in any fashion, that is an
> improvement in the end user experience, however achieved.  
> 
> Warmest Regards;
> 
> Jack Rickard
> 
>  ===================================================================
> Jack Rickard                                    Boardwatch Magazine
> Editor/Publisher                    8500 West Bowles Ave., Ste. 210
> jack.rickard at boardwatch.com                     Littleton, CO 80123
> www.boardwatch.com                             Voice: (303)973-6038
> ===================================================================
> 
> 
> 
> 
> ----------
> > From: Craig A. Huegen <c-huegen at quadrunner.com>
> > To: Jack Rickard <jack.rickard at boardwatch.com>
> > Cc: Peter Cole <Peter.Cole at telescan.com>; nanog at merit.edu;
> marketing at keynote.com
> > Subject: Re: Keynote/Boardwatch Internet Backbone Index  A better test!!!
> > Date: Friday, June 27, 1997 2:08 PM
> > 
> > On Fri, 27 Jun 1997, Jack Rickard wrote:
> > 
> > ==>Not an entirely whacky concept actually.  The way hot potato routing
> works,
> > ==>this could actually be a "purer" test I suspect of the network
> internally
> > ==>and a purer test of connectivity of any network to all others cum
> Internet.
> > 
> > The article says that you're measuring backbone provider performance, yet
> > you're including:
> > 
> > * carrier's web server location
> > * carrier's web server performance
> > 
> > ==>Keynote does a "Top 40" study of 40 popular web sites and I believe
> they
> > ==>make the results available on their web site.  It is interesting to
> observe
> > ==>performance variations of the network as a whole over time.  Other
> than
> > ==>that, we don't have much interest in it.  It is indicative of no
> specific
> > ==>network, but of the Internet in general.
> > 
> > /cah