HE.net, Fremont-2 outage?

Wed Nov 4 07:09:48 UTC 2009

Joe Greco wrote:
>> Jeffrey Lyon wrote:
>>     
>>> FWIW: http://www.he.net/releases/release18.html
>>>       
>> No date on that 'press release' but the way back machine helps put it
>> somewhere in 2002. A lot of good this "Alameda" sized generator has done
>> recently...
>>
>> http://web.archive.org/web/*/http://www.he.net/releases/release18.html
>>     
>
> 2MW isn't super huge or anything.  I would expect that, given the size
> I have been led to believe HE is, they've got a lot more than that now.
>
> My memory is that Alameda isn't huge, but it isn't small either.  I'm
> not sure ..  ah, here
>
> http://www.reuters.com/article/pressRelease/idUS179594+03-Apr-2009+BW20090403
>
> peak 70MW
>
> I'm not sure what the basis for the claim is that a 2MW generator is
> "large enough to power the entire city of Alameda" ...  2MW gensets are
> common enough in this business and it's possible to burn through 2MW in
> a few hundred racks.  It isn't *that* much power.
>
> A more conventional comparison might be to something like a hospital; one
> of our local hospitals installed a 1.25MW generator which, IIRC, powers
> all critical circuits.
>
> http://hhenergyservices.com/electrical/photos.php?category_id=2845&subcategory_id=5027&id=196&number=7
>
> Sometimes it is easier to picture things that way.
>   

Regardless of generator sizing issues or disparities, if the ATS fails, 
then no amount of grid or generator power will keep the cabinets juiced up.

Since this is the second time in recent history that this building has 
experienced a short power outage caused by ATS flakiness, perhaps 
keeping a small UPS in the cabinet isn't such a bad idea? Even if the 
distribution switches/routers lose power, at least the servers wouldn't 
have to go through fscks and DB integrity checks due to unplanned power 
loss, and the recovery time would be significantly faster.

Hell, for a 5 minute power outage, some of my services were down for 20 
minutes. I'll happily take a 75% reduction in downtime for the cost of a UPS
, though clearly redundancy across more reliable datacenters is a better 
solution.

> ... JG
>   