Never push the Big Red Button (New York City subway failure)

Jay R. Ashworth jra at baylink.com
Sun Sep 12 03:23:41 UTC 2021


----- Original Message -----
> From: "Sean Donelan" <sean at donelan.com>

> NEW YORK CITY TRANSIT RAIL CONTROL CENTER POWER
> OUTAGE ISSUE ON AUGUST 29, 2021
> Key Findings
> September 8, 2021
> 
> https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf
> 
> Key Findings
> [...]
> 
> 3. Based on the electrical equipment log readings and the manufacturer’s
> official assessment, it was determined that the most likely cause of RCC
> shutdown was the “Emergency Power Off” button being manually activated.

I don't even *do* datacenter for a living, and I know that when you hit the
Molly button, 

1) A Klaxon goes off in the Data Center -- one that sounds *different* from
the Halon Klaxon, in both cadence and tone (just for a couple bursts), and

2) Yellow rotating beacons turn on, and stay on while you're on Emergency Power.

Yes, real honest-to-ghod *rotating mechanical beacons*, none of this flashing LED
crap.

Clearly, it's important that the use of Emergency Power be annoyingly noticeable.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra at baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates       http://www.bcp38.info          2000 Land Rover DII
St Petersburg FL USA      BCP38: Ask For It By Name!           +1 727 647 1274


More information about the NANOG mailing list