Traffic locality and other questions

Mon Sep 22 21:50:04 UTC 1997

Traffic locality has long been a concern of mine, since one would expect
local public library traffic to be dominated by local users.  What's
interesting is over the last five years the network topology has become
less local.  Once upon a time, almost every site within a geographic
region 'bought' service from a single mid-level provider.  MIDNET served
the midwest, MERIT served Michigan, SURANET served the southeast,
BARRNET served the San Francisco bay area, and so on.  The concept of the
regional provider dropped out of favor as MCI, Sprint and UUNET entered
the marketplace with significantly lower charges than the regional providers.

Now network topology, like the airline industry, favors tail circuits
feeding into large hubs.  With facilities based providers its not uncommon
to see a >1,000 mile back-haul for the local loop.  Even if both end-users
bought service from the same provider, they may end up with service from
hubs thousands of miles away from each other.  For example, its not uncommon
for St. Louis customers to be served from Chicago, Kansas City, or Fort Worth.

This is most pronounced in the individual user marketplace.  Even if the
individual user dials a local phone number, often their connection is
hauled to someplace like Northern Virginia (AOL) before heading out on
the Internet.  So the best performance may come from a provider  with
connections closer to Northern Virginia than to some place near the user's
dialup.

What's also interesting is the asymmetry between traffic generators and
traffic consumers.  Although overall DRA has a even mix of inbound and
outbound traffic.  I see tremendous inbalances with individual providers.
But it is very difficult to tell who is getting a 'free-ride' from whom.
Other than SYN-flood attacks, and the dreaded default route, traffic only
flows if there is some kind of customer demand on both sides.

I have some questions whether it is better to aggregate traffic into
a single huge flow, or if it is better to have lots of smaller paths.
Would I be better buying a single cross-country OC3 with one provider,
or trying to do a deal for several T1's with Turner, Gannett, or other
places public libraries are interested in using?  Over the last five
years, I've found building small bypasses work better.  But maybe things
have changed and I can meet end-to-end performance requirements by
other means.

I also have a question about how consolidated the traffic really is.
If I only bought circuits from the top three providers, how much of
our traffic would that really cover.  What if I bought circuits from
the top 10 providers.  Perhaps the public library market is an aberration,
but I see much less traffic aggregation than others report.  I use both
'managed' and 'un-managed' connections in both 'transit' and 'peer'
provider relationships.  My business relationship with the next-hop
provider seems to have very little correlation with the traffic flows.

DRA's top 20 traffic generators/sinks by end AS, alphabetical order

   AOL, BBN, CONCERT, DIGEX, DISI, EDS, GANNETT, IACNET, ISI, JVNC, MCI,
   MERIT, NETCOM, OARNET, PSI, SPRINT, SUPERNET, TURNER, UNISYS, UUNET

The top 20 end ASs are extremely volatile.  NASA Science Internet can leap
to the top for a week, e.g. Pathfinder; or IBM jumps during the Olympics,
or UK providers the week following Princess Diana's death.  Since most
of the long-term large traffic generators are multi-homed among several
providers, a very small network topology change can dramatically shift
traffic between network backbone providers.

Percentage of traffic destined for the top 10 next-hop ASs, includes
transit ASs so some next-hop traffic may be inflated by traffic destined
to another provider via the transit provider.  Since most peering agreements
prohibit the release of identifiable information, I'm not identifying
which provider goes with what percentage.

	14.4%
	13.4%
	12.0%
	 7.9%
	 3.2%
	 1.9%
	 1.8%
	 1.2%
	 1.1%
	 1.1%

BGP isn't very good at showing 'other' transit paths to networks.  Its
like adding another lane to a road, traffic that didn't exist before
appears.  The next-hop rankings are less volatile, but over time they
do show significant trends.  In the last year, the traffic among next-hops
has become flatter at the top, and the tail has become longer.  The
opposite of what I would expect in a market experiencing consolidation.
What I find interesting is the rankings of traffic flows I see don't
match with what the pundits rank as the largest network providers.  I
don't know what that means though.
-- 
Sean Donelan, Data Research Associates, Inc, St. Louis, MO
  Affiliation given for identification not representation