<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Now I'm curious... in all of the DCs and COs I've worked in - to the best of my knowledge, I haven't personally tested this! - the EPO button does
<b>not</b> switch to emergency power. It turns off ALL equipment power in the space - no lights, no klaxons, nothing. In simpler setups, the EPO is connected to the UPS so anything plugged in to the UPS does dark instantly. In one DC I'm familiar with,
the EPO switch kills all the UPS output <b>and</b> uses several relays to kill commercial power at the same time.<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
In some, the room lights were not covered by the EPO switch, in some they were. Emergency exit lamps will continue to be lit, as they have internal batteries, and are required by building/fire code.<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Is it (somewhat) common for an EPO switch to only disconnect commercial power and leave local redundant power live? What sort of facilities would have this?<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
-Adam<br>
</div>
<div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b style="text-align:start; background-color:rgb(255,255,255)"><span style="margin:0px; font-size:10pt; color:rgb(68,84,106)">Adam Thompson</span></b><span style="margin:0px; font-size:9pt; color:rgb(68,84,106); text-align:start; background-color:rgb(255,255,255)"><br>
Consultant, Infrastructure Services<br>
<img class="EmojiInsert" alt="1593169877849" style="margin:0px; max-width:100%; height:auto" data-outlook-trace="F:2|T:2" src="cid:594e8d00-4354-4627-8fed-becb815c0769"><br>
100 - 135 Innovation Drive<br>
Winnipeg, MB, R3T 6A8<br>
(204) 977-6824 or 1-800-430-6404 (MB only)<br>
<a href="mailto:athompson@merlin.mb.ca" target="_blank" rel="noopener noreferrer" style="margin:0px"><span style="margin:0px">athompson@merlin.mb.ca</span></a><br>
<a href="http://www.merlin.mb.ca/" target="_blank" rel="noopener noreferrer" style="margin:0px"><span style="margin:0px">www.merlin.mb.ca</span></a></span><br>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> NANOG <nanog-bounces+athompson=merlin.mb.ca@nanog.org> on behalf of Jay R. Ashworth <jra@baylink.com><br>
<b>Sent:</b> September 11, 2021 22:23<br>
<b>To:</b> nanog <nanog@nanog.org><br>
<b>Subject:</b> Re: Never push the Big Red Button (New York City subway failure)</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">----- Original Message -----<br>
> From: "Sean Donelan" <sean@donelan.com><br>
<br>
> NEW YORK CITY TRANSIT RAIL CONTROL CENTER POWER<br>
> OUTAGE ISSUE ON AUGUST 29, 2021<br>
> Key Findings<br>
> September 8, 2021<br>
> <br>
> <a href="https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf">
https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf</a><br>
> <br>
> Key Findings<br>
> [...]<br>
> <br>
> 3. Based on the electrical equipment log readings and the manufacturer’s<br>
> official assessment, it was determined that the most likely cause of RCC<br>
> shutdown was the “Emergency Power Off” button being manually activated.<br>
<br>
I don't even *do* datacenter for a living, and I know that when you hit the<br>
Molly button, <br>
<br>
1) A Klaxon goes off in the Data Center -- one that sounds *different* from<br>
the Halon Klaxon, in both cadence and tone (just for a couple bursts), and<br>
<br>
2) Yellow rotating beacons turn on, and stay on while you're on Emergency Power.<br>
<br>
Yes, real honest-to-ghod *rotating mechanical beacons*, none of this flashing LED<br>
crap.<br>
<br>
Clearly, it's important that the use of Emergency Power be annoyingly noticeable.<br>
<br>
Cheers,<br>
-- jra<br>
-- <br>
Jay R. Ashworth Baylink jra@baylink.com<br>
Designer The Things I Think RFC 2100<br>
Ashworth & Associates <a href="http://www.bcp38.info">http://www.bcp38.info</a> 2000 Land Rover DII<br>
St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274<br>
</div>
</span></font></div>
</body>
</html>