backbone transparent proxy / connection hijacking

Fri Jun 26 23:31:06 UTC 1998

I wanted to take a moment to respond to this thread which has gotten
somewhat inflamed. The problems being highlighted are not
new or unknown and there are standard remedies in use by
Inktomi's Traffic Server customers and other users of transparent
caching. In fact, the two posters reporting concrete problems have
both already had them swiftly remedied.

Transparent caching brings with it significant benefits to
the ISPs and backbones who deploy it, but also to the
dialup users, corporate customers and downstream ISPs who
utilize those links. Cached content is delivered accurately
and quickly, improving the experience of web surfing.
Further, caching helps unload congested pipes permitting increased
performance for non-HTTP protocols. Many people believe that
large-scale caching is necessary and inevitable in order to scale
the Internet into the future.

I will spend a few paragraphs talking about each of the concerns
which have been expressed in this thread. Roughly, I think they
are the following: disruption of existing services, correctness
of cached content, and confidentiality/legal issues with
transparent caching. We take all of these issues very seriously
and have had dedicated resources in our development and technical
support groups addressing them for some time.

The center of this debate concerns the rare disruption of
existing services which can occur when transparent caching is
deployed. Two concrete examples of this have been cited on this
list: access to a Cybercash web server and access from an old Netscape
proxy server. Both of these incidents were swiftly and easily
corrected by the existing facilities available in Traffic Server.

The Cybercash server performed client authentication based on
the IP address of the TCP connection. Placing a proxy (transparent
or otherwise) in between clients and that server will break
that authentication model. The fix was to simply configure Traffic
Server to pass Cybercash traffic onwards without any attempt to
proxy or cache the content.

The second example was of a broken keepalive implementation in
an extremely early Netscape proxy cache. The Netscape proxy
falsely propagated some proxy-keepalive protocol pieces, even
though it was not able to support it. The fix was to configure
Traffic Server to not support keepalive connections from that
client. Afterwards, there were no further problems.

These two problems are examples of legacy issues. IP-based
authentication is widely known to be a weak security measure.
The Netscape server in question was years old. As time goes
on, there will be a diminishing list of such anomalies to deal
with. Inktomi works closely with all of our customers to
diagnose any reported anomaly and configure the solution.

Beyond that, to scale this solution, Inktomi serves as a
clearinghouse of these anomaly lists for all of our customers.
A report from any one customer is validated and made available
to other Traffic Server installations to preempt any
further occurrences.

Inktomi also conducts proactive audits both inside live Traffic
Servers and via the extensive "web crawling" we perform as part
of our search engine business. The anomalies discovered by these
mechanisms are similarly made available to our customers.

The second issue being discussed is the correctness of cached
content. Posters have suggested mass boycotting of caching by
content providers concerned with the freshness of their content.
Most content providers have no such concerns, frankly. The problem
of dealing with cached content is well understood by publishers
since caching has been in heavy use for years. Every web browser
has a cache in it. AOL has been caching the lion's share of
US home web surfers for years. For more information on the ways
in which publishers benefit from caching see our white paper
on the subject of caching dynamic and advertising content:
http://www.inktomi.com/products/traffic/tech/ads.html

And finally, there has been confusion concerning the
confidentiality and legal issues of transparent caching.
Transparent caching does not present any new threat to the
confidentiality of data or usage patterns. All of these issues
are already present in abundance in the absence of caching.
Individuals responsible for managing networks will have to weigh
the advantages of caching against these more nebulous
considerations. We, and many others looking towards the future
of a scalable Internet, are confident that caching is becoming an
integral part of the infrastructure, and provides many benefits to
hosters, ISPs, backbones and surfers alike.

Paul Gauthier

-- 
Paul Gauthier,                                   (650)653-2800
CTO, Inktomi Corporation                  gauthier at inktomi.com