East Coast Earthquake 8-23-2011 - comment and a bit of a Christchurch Telco report :)

Don Gould don at bowenvale.co.nz
Wed Aug 24 22:53:08 UTC 2011



On 25/08/2011 12:27 a.m., Scott Morris wrote:
> Also, the quake on the east coast was much closer to the surface than
> most west coast quakes, which could account for the feeling.
>
> Scott (not a geologist)

/me is also not a geologist, but does live in the east part of 
Christchurch, New Zealand.  Our CBD has been closed for 6 months now as 
a result of a 6.3 on February 22, 2011.

A number of factors effect impact on buildings from our experience. 
Depth is a big issue.  A quake 80km deep at 5.8 will have far less 
impact than one 1km deep.  When I read yours was 1km deep, I started 
looking for the impact stories.

Direction of the movement is also a big issue.  In our case the ground 
was going up and down under our tall(ish) buildings faster than the 
buildings.  The ground would start coming up again before the building 
had finished coming down again from the last movement cycle.  This just 
smashes buildings to bits.

Our tallest building is 26 stories, will take a year to bring down and 
is on a lean.  Interesting most of the damage was caused by the building 
next to it being hammered into the side.  We have many ~10 story 
buildings to come down.

Anyway enough about buildings... here's some comment on networks that I 
thought some might find interesting...


Networks - Mobile

Mobile traffic does go nuts.  We have an average (iirc) of less than 30 
minutes voice traffic per month per user.  So out networks simply are 
not designed with a large load in mind.

We also don't have the 'confirmation' aspect of the sms (texting) 
protocol turned on.  This means that the senders mobile phone doesn't 
know if the message has been delivered to the receiver.

In our case, we have 3 mobile networks in Christchurch.  We discovered 
that we simply didn't have enough inter carrier capacity provisioned for 
sms traffic.  While users could send messages 'same network to same 
network', they couldn't send them 'off network'.

We also have full number portability.  This means that senders don't 
know which network they're even sending messages to any more.

+64 021 - Vodafone
+64 022 - 2Degrees
+64 027 - Telecom

Fail - not any more.

The up shot is that when disaster struck us, text/sms messages were not 
getting though and now one knew what the problem was.

So people started to attempt phone calls (on a network provisioned for 
an average load of ~30 minutes a month - you can see how this is heading 
down hill fast!).

The really ironic part of the lack of inter-carrier capacity for sms was 
that the 3G data capacity never failed for me.  So while I couldn't send 
a 200byte message, I could send a 300kbyte email with photos.

!!!POWER!!!

Battery life also quickly becomes a problem for both network operator 
and user.  As towers start shutting down, phones move to more distant 
towers, which mean everything uses more power.

Telco's are then presented with the problem of getting generators to 
towers.  "Remember your chains and padlocks!" - when disaster strikes, 
idiots think stealing telco generators is cool.  You don't want to have 
to revisit a tower just to replace the generator.

Home users quickly realise they have no way to recharge their mobiles. 
New, cheap, smart phones only last a day and even less when they're 
trying to talk to distant towers.  (You should see the pile of hand held 
'crank' torch/radio/mobile phone chargers that our local hardware store 
now has!)

People are also asking about inverters on local message boards.

Population Movement...

Another problem we've found is that population movement causes load 
issues.  Thousands of people left the city area to towns up to 3 hours 
away to escape the aftershocks.  They take their phones and mobile 
broadband and then spent lots of time calling back into the city to 
local friends and family.  Suddenly everyone's doing much more calling 
from small towns back into the city over the mobile network than normal.


Networks - Fixed

Our fixed lined networks did stay up reasonably well.  (We have two 
separate copper networks in 50% of the city - Telecom & TelstraClear, 
and one incumbent who covers the whole city - Telecom.)

However, the power went out (in many cases simply because the street 
side transformers have sensors in them to detect if the oil reservoir 
has moved.  The earth quakes trip the sensors and they either go out for 
a while or you have to wait until someone comes and resets them I think 
- either way, it resulted in 9 hours with no power for me on the 22nd).

Many many people use cordless phones and don't have a non-powered/corded 
phone any more.

After the 22nd we had a national campaign to get old corded phones to 
Christchurch.  5,000 were collected.  (Now when you consider the country 
has a population of 4m and Christchurch has ~360k, 5,000 phones is quite 
a few).

We are moving forward into an FTTN/H world.  This means that homes need 
power to keep voip units up.  Simple rule - when disaster strikes, 
mobile is going to be 'it', so make sure you've got enough capacity 
provisioned!

Our road side FTTN cabinets only have battery life for between 4 to 8 
hours.  Each cabinet services ~200 customers.  So you can see a new 
problem unfolding... how do you get generators to those?  The FTTN nodes 
currently do midpoint injection, so the PSTN stayed going but DSL failed 
after time, expect where the 'E' side cables where damaged.

The newer TelstraClear fixed network is a combination HFC/UTP POTS 
network delivered over head or in street ducting.  Interestingly the HFC 
didn't miss a beat.  The POTS did eventually drop out, and when it came 
back on line, the system only seemed to have a basic configuration and 
the lines wouldn't support appliances like dialup EFTPOS machines any 
more.  (This was fixed a few days later.) - oh how I long for a total IP 
world! :)

EFTPOS is very important in local socio-economic high crime areas. 
Small local store owners start to stress when they start to collect to 
much cash in the till as it makes them a target.  EFTPOS means they can 
'give out cash' at the same rate they collect it.  Many people on public 
benefits, such as the old age pension, also don't have cash and use a 
plastic card to buy something as simple as milk.

While people had 'money in the bank', they couldn't get to it to spend 
it end they needed to.  IP EFTPOS means you can transfer the terminal 
from a fixed service to another network or mobile solution quickly.

My local store owner only had fixed lines from the one provider whos 
service failed, so I had to string cables down 4 buildings to get a 
working phone service from 'the other network'.  (We were lucky that one 
building in the block did still have a working service from 'the other 
provider'.  $5 difference a month means that 70% of customers in the 
area are using the cheaper provider).

Ground Cables ---

The older Telecom network is 'direct bury' and the cables in my area are 
~60 years old.

This causes a number of problems.  With 7,000 after shocks since our 7.1 
quake on Sept 4, 2010, the cables are getting pulled apart and pulled 
away from street side termination.  We now have holes on our road 
because of earthquake damage and holes in our foot paths from telco 
contractors digging up bits of cable all over the suburb to trace faults.

While the old cables are armoured, you only have to touch them with a 
digger and you get 4 nice cable techs parked out side your house for 2 
days drinking your coffee while they dig up and fix 85 pairs.  (Which I 
might note only supplies service to 1 customer now)

City wide we've had many buildings pulled down.  This also causes 
problems where cables run across buildings.  Local techs told me that 
they've had many call outs where the cable to an adjacent building has 
simply been removed when a building is demolished in a hurry (because 
it's likely to fall on someone if there's another aftershock.)

We also had inter-exchange load issues.  People could call 'out of town' 
but not exchange to exchange - oh how I long for a total IP world! :)





-- 
Don Gould
Christchurch, New Zealand





More information about the NANOG mailing list