An appeal for more bandwidth to the Internet Archive

Jonah Edwards jonah at
Wed May 13 19:00:10 UTC 2020

Thank you Jared & Tim!  Fortunately the recent upgrade has gotten us 
some overhead in the short term -- we generally run very close to the 
edge, capacity-wise, in order to get the most out of what we have, so a 
sudden unexpected burst in demand pinned us for a bit there.  I expect 
more upgrades to be necessary in the near future, particularly on the 
egress bandwidth and long-haul datacenter interconnect fronts, and would 
love to talk off-list about help with peering/pipes in the Bay Area and 
100G gear especially.

To answer a couple sibling thread questions, traffic is pretty 
legitimate -- lots of users accessing lots of content.  We're fine with 
folks downloading in bulk and most who do use our torrent functionality 
to do so.  We have a broad view of regionality of visitors, but we very 
intentionally don't keep specific usage records (after all, we are a 
library, and we don't track our patrons -- this also precludes the use 
of most commercial CDNs).  See e.g. 
for some info.

As far as colocation costs, we run our own datacenters in the Bay Area, 
where most of our on-site staff lives and where fully ambient cooling is 
possible year-round.  Our presence in 200 Paul is a terminus for our 
dark fiber and a place to make XCs -- actually only a single rack.

Thanks again for the all the appreciation and help -

Jonah Edwards
jonah at

On 2020-05-12 09:33, Tim Požár wrote:
> Jared...
> Thanks for sharing this.  I was the first Director of Operations from
> '96 to '98, at was was then Internet Archive/Alex.  I was the network
> architect back then got them their ASN and original address space.
> Folks may also know, I help start SFMIX with Matt Peterson.
> A bit more detail in this...  Some of this I got from Jonah Edwards
> who is the current Network Architect at IA.  Yes, the bottle neck was
> the line cards.  They have upgraded and that has certainly helped the
> bandwidth of late.
> Peering would be a big help for IA. At this point they have two 10Gb
> LAG interfaces that show up on SFMIX that was turned up last February.
> Looking at the last couple of weeks the 95th percentile on this 20Gb
> LAG is 3 Gb.  As they just turned up on SFMIX, they are just starting
> to get peers turned up there. Eyeball networks that show up on SFMIX
> are highly encouraged to start peering with them.  Alas, they are v4
> only at this point.
> Additionally, if folks do have some fat pipes that can donate
> bandwidth at 200 Paul, I am sure Jonah won't turn it down.
> Tim
> On 5/12/20 4:45 AM, Jared Brown wrote:
>> Hello all!
>> Last week the Internet Archive upgraded their bandwidth 30% from 47 
>> Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely 
>> solid green graph showing how usage grows vertically as each interface 
>> comes online until it too is 100% saturated. Looking at the graph 
>> legend you can see that their usage for the past 24 hours averages 
>> 49.76G on their 50G of transport.
>> To see the pretty pictures follow the below link:
>> Relevant parts from the blog post:
>> "A year ago, usage was 30Gbits/sec. At the beginning of this year, we 
>> were at 40Gbits/sec, and we were handling it. ...
>> Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our 
>> network infrastructure’s ability to handle it.  So much so, our 
>> network statistics probes had difficulty collecting data (hence the 
>> white spots in the graphs).
>> We bought a second router with new line cards, and got it installed 
>> and running (and none of this is easy during a pandemic), and 
>> increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak.   
>> And we are handling it better, but it is still consumed."
>> It is obvious that the Internet Archive needs more bandwidth to power 
>> the Wayback machine and to fulfill its mission of being the Internet 
>> library and the historic archive of our times.
>> The Internet Archive is present at Digital Realty SFO (200 Paul) and a 
>> member of the San Francisco Metropolitan Internet Exchange (SFMIX).
>> I appeal to all list members present or capable of getting to these 
>> facilities to peer with and/or donate bandwidth to the Internet 
>> Archive.
>> I appeal to all vendors and others with equipment that they can donate 
>> to the Internet Archive to contact them so that they can scale their 
>> services and sustain their growth.
>> The Internet Archive is currently running 10G equipment. If you can 
>> help them gain 100G connectivity, 100G routing, 100G switching and/or 
>> 100G DWDM capabilities, please reach out to them. They have the 
>> infrastructure and dark fiber to transition to 100G, but lack the 
>> equipment. You can find the Internet Archive's contact information 
>> below or you can contact Jonah at the Archive Org directly either by 
>> email or via the contact information available on his Twitter profile 
>> @jonahedwards.
>> You can also donate at
>> The Internet Archive is a 501(c)(3) non-profit. Donations are  
>> tax-deductible.
>> Contact information:
>> Volunteering:
>> Disclaimer: I am not affiliated with the Internet Archive. Nobody 
>> asked me to write this post. If something angers you about this post, 
>> be angry at me. I merely think that the Internet Archive is a good 
>> thing and deserves our support.
>> Jared

More information about the NANOG mailing list