Internet impact of Apple Tiger

Fri Apr 29 14:02:02 UTC 2005

Some food for thought:

-- 
http://www.gigaom.com/2005/04/28/rss-tiger-safari-and-the-bandwidth-bottlene
ck/

RSS, Tiger Safari and the Bandwidth Bottleneck

In less than 48 hours many of us will be installing Tiger OS-X and with it a
brand new Safari browser that can read and display RSS feeds in a simple
easy to understand manner. That upgrade while great for the consumers, could
come as a big shocker for those blogs whose feeds are included as part of
Safari¹s default starter package. Infact it could be the biggest stress test
for RSS thus far!

Most RSS readers are set to poll for updates every hour, and imagine when
half-a-million Tiger Safari users who start hitting a server at the same
time, pulling down RSS updates, because they have not changed the default
settings. Server meltdown? Or an unintended denial of service? Apple says
that most of the default feeds are going to be major news sites like CNN.
New York Times, and LA Times. At this time they are not including any
personal blogs as part of the default list. Even for them it is not going to
be easy.

Lets say if one of these news operations updated their site once an hour and
each update results in a nominal 5 kilobytes of RSS generated data, then
500,000 simultaneous Safari users polling at top of the hour would mean a
total data transfer of over 2 gigabytes per hour. Times 24, and you have
over 48 gigabytes of data transfer every day  just from Safari users alone.
What if more than a million Tiger Safaris were on the loose. Oh boy! While
an addition 48 gigabytes of traffic a day or 1.4 terabyte a month is not
that much for large sites, but it will add up.

Admittedly, since I don¹t have Tiger yet, not sure if Safari RSS does
time-based check (every hour at :15) or checks related to when the
computer/browser is started, which is relatively random and what other feed
readers do. Clearly this is an imaginary scenario, but it could happen. So
what¹s the fix? ³I certainly hope that Safari does conditional GET. I can¹t
imagine it doesn¹t but I could be wrong,² says Brent Simmons, founder and
the man behind hit feed reader, Net News Wire, ³With conditional GET you
download the feed only if it¹s different from the last time you downloaded
it ‹ this cuts way down on bandwidth use.² (More on conditional GET.)
³Conditional GET ‹ which NetNewsWire and most other aggregators support ‹ is
hugely important,² says Simmons. But even that can go that far, since most
of these news operations churn out headlines with monotonous regularity.

Long term, I think RSS is going to become a clear bandwidth hog, unless the
RSS people decide and come-up with an intelligent way to fix this problem. I
have been tracking my own bandwidth consumption and RSS is just sucking up
gigabytes like a parched man on a hot summer day. Some say that randomizing
the whole RSS polling process is the answer.

How about randomizing the whole RSS polling process? Instead of pulling down
RSS feeds every hour, let the feeds download randomly. Okay that will help
distribute the loads on the servers more evenly, but that still doesn¹t
resolve the issue of inefficient use of network resources, especially for
those who pay for those kind of things. Suggestions?

Scott Rafer, CEO of Feedster says, ³ISPs can start caching feed URLs but if
they do it with cached times of more than 10 min, then people will route
around the caches.²

  Om Malik on Broadband 4/28/05 2:57 AM Om Malik Connected Home, Wired,
Featured Comments