Big Temporary Networks

Thu Sep 13 23:17:44 UTC 2012

On 13 September 2012 22:13, Jay Ashworth <jra at baylink.com> wrote:
> ----- Original Message -----
>> From: "Josh Baird" <joshbaird at gmail.com>
>> Besides this, we have a fairly beefy box that handles DNS and DHCP and
>> basic firewalling.
>
> Have you had to/been able to haul in your own bandwidth to feed it?  What
> class?  (Real DS3/OC1/OC3, FiOS/HFC, something else?)

Two weekends ago EMFCamp took place north of London in Milton Keynes,
the UK’s first maker weekend long festival, ran along the same lines
as CCC / HAR2009 etc.

A small team of us designed the infrastructure for it, we started at
the end of May, 3 months in advance. The CCC noc team in Germany were
kind enough to lend us their event /19 + /48 + ASN, we built a
temporary network spanning from Telehouse East in London Docklands up
to a local data centre (Pulsant) in Milton Keynes.

Pulsant sponsored us with a 1gb/s L2 circuit from Telehouse to Milton
Keynes, we placed a router (c7202+npe-g2) in each decenter. We took on
transit in both sites and had temporary membership to LONAP in
Telehouse where we connected to their route server for v4,v6 peering
and even multicast.

Biggest cost was the 2 mile link from the dc back to the festival
site, we rented 2 portable 30m trailer mounted masts. A microwave
company loaned us some DragonWave kit which ran on 18ghz at 385mb full
duplex, this was our primary link and they applied for a UK OFCOM temp
telco license for this on our behalf. We also bought a pair of
Ubiquiti Nanobridge M5’s for backup, running at about 100mb.

We didn’t firewall anything, users were made aware what they were
connecting to, there were no passwords on the SSID’s, we had no agenda
to monitor traffic. We published abuse email addresses and a number
that people could call if required and we would act on it (the RIR
contacts for the address space were updated too)

Our NOC:
http://www.flickr.com/photos/nottinghack/7929611918/
https://dl.dropbox.com/u/74717/2012-08-30%2017.40.26.jpg
http://www.flickr.com/photos/russss/7909193016/
http://www.flickr.com/photos/nottinghack/7929909834/

Onsite core and servers
http://www.flickr.com/photos/nottinghack/7929611592/
http://www.flickr.com/photos/andy_d/7902260210/

For wireless we deployed a pair of Cisco wireless controllers, all the
APs were lightweight and RF allocation was easily managed centrally.
https://twitter.com/emfnoc/status/241944863887749121/photo/1

Just like CCC + HAR we deployed portaloo’s / datenklo around the
campsite and campers connected up to them for power and Ethernet:
http://www.flickr.com/photos/ne0hack3r/7924490940/
http://www.flickr.com/photos/je4d/7924689482/

Sort out kit configuration out well in advance, really glad we did as
we spent far longer getting the mast and microwave kit aligned that we
thought. Switches, servers were all configured before arriving so we
just unloaded and connected things up according to the plan. Avoid
NAT’ing anything, speak to a friendly ISP and borrow some address
space. We split DNS resolvers, DHCP, monitoring VMs across 3 separate
VM hosts just in case one had a hardware failure, don't rely on a
single server box.

Do it properly and attendees will be happy:
https://twitter.com/Ash_Force/status/242067006537474048
https://twitter.com/markphelan/status/241896897290309633
https://twitter.com/je4d/status/242386884276396032

Our slides are here (warning 50mb)… http://www.natmorris.co.uk/camp_network.pdf

Get a team on board to help out, ours rocked!

-- 
Nat

http://natmorris.co.uk
http://twitter.com/natmorris