rack power question
Barton F Bruce
barton at gnaps.com
Sat Apr 5 10:39:15 UTC 2008
>> A close second might be liquid cooled air tight cabinets with the
>> heat exchangers (redundant pair) at the bottom where leaks are less of an
>> issue (drip tray, anyone? :) )...
> Something like what you suggest has been around for a year or two now,
> though using liquid CO2 as the coolant. It doesn't require particularly
> tight cabs.
Is anyone using these over here?
This is a far more significant strategy that simply using an alternative to
water to carry the heat from the cabinets.
The game is PHASE CHANGE, but unlike our traditional fairly complicated
refrigeration system systems with oil return issues and artificiaally high
head pressures simply to have a 100PSI MOPD to keep full flow through the
TXV (even with low ambients outside) this is in its simplest form viewed as
a PUMPED LIQUID heat pipe system, where there is no need for large pressure
drops as the fluid goes around the loop. Your pump only has to cover piping
loses and any elevation differences between the COLO space and the central
There is NO insulation at all. The liquid being pumped out to finned coils
on the back of each cabinet is at room temperature and as it grabs heat from
the cabinet exhaust air (which is very efficient because you have it HOT and
not needlessly undiluted with other room air) some of the liquid flashes to
gas and you have a slurry that can easily be engineered to handle any size
load you care to put in the rack. The more heat you add, the more gas and
the less liquid you get back, but as long as there is still some liquid, the
fluid stream is still at the room temperature it was at before entering the
coil. It is perfectly happy trying to cool an empty cabinet and does not
over cool that area, and can carry as much overload as you are prepaired to
pay to have built in.
At the central equipment, the liquid goes to the bottom of the receiver
ready for immediate pumping again, and the gas is condensed back to liquid
on cold coils in this receiver (think of a large traditional shell and tube
heat exchanger that also acts as a receiver and also a slight subcooler for
the liquid). The coils can be DX fed with any conventional refrigerant, or
could be tied to the building's chilled water supply. Parallel tube bundles
can provide redundant and isolated systems, and duplicating this whole
system with alternate rows or even alternate cabinets fed from different
systems lets you function even with a major failure. Read about their
scenarios when a cooling door is open or even removed. The adjacent cabinets
just get warmer entering air and can easily carry the load. Enough 55 degree
ground water in some places might even let you work with a very big shell
and tube condenser and NO conventional refrigeration system at all.
If you have every single cabinet packed full, having just two systems each
needing full double+ capacity would not be as good as having 3 or 4
interleaved systems, but that is simply a design decision, but one that can
be partially deferred. Pipe for 4 interleaved isolated systems, and then run
the ODD ones into one condensing/pumping system, and the EVEN ones into
another. As cabinets fill, and as dollars become available for paranoia, add
the other central units and flick a few normally padlocked preprovisioned
valves and your are done. The valves stay for various backup strategies. You
can accidentally leak some CO2 from one system to another and then sneak it
back. There are NO parallel compressor oil return issues, just a large range
between min and max acceptible charges of CO2.
The big problem is that CO2 at room temperature is about 1000 PSI, so all
this is welded stainless steel and flexible metal hoses. There need not be
enough CO2 in any one system to present any suffocation hazard, but you DO
want to be totally aware of that in the design.
Unlike regular refrigerants, liguid CO2 is just dirt cheap, and you just
vent it when changing a finned rear door - each has its own valves at the
cabinet top main pipes.. You just go slowly so you don't cover everything or
anyone with dry ice chips.
Here is another site hawking those same Trox systems:
Over in Europe they are talking of a demo being easliy done if you already
have chilled water the demo could use.
A recent trade mag had a small pumped heat pipe like R134a system for INSIDE
electronic systems - a miniature version of these big CO2 systems. Heat
producing devices could be directly mounted to the evaporator rather than
use air cooling fins or a water based system, and the condenser could be
above. or below or wherever you need to put it and could function in
arbitrary positions in the field. And no heat pipe wicks needed. The fully
hermetic pump has a 50K hour MTBF and is in this case pumping R134a. The
pump looked like one of those spun copper totally sealed inline dryers. I
suspect it was for high end computing and military gear, and not home PCs,
but clearly could move a lot of heat from very dense spaces. Parker is so
sprawling, I can't now seem to readily find the division that makes that
pump and that wants to design and build the whole subsystem for you, but I
bet we will be seeing a lot of these as power density goes up.
And on a totally seperate rack power topic, (code issues aside - that can be
changed given need and time), I would LOVE to see some of these 100 - 250V
universal switching supplies instead made to run 100 - 300V so they could be
run off 277V.
It is silly to take 277/480 through wastefully heat producing Delta-Wyes
just to get 120/208 to feed monster power supplies that really should be fed
with what most building already have plenty of. The codes could easily
recognize high density controlled access data centers as an environment
where 277V would be OK to use for situstions where it isn't ok now. Small
devices sharing the same cabinets should all be allowed to also use 277V.
We used to have cabinets of 3 phase 208Y fed Nortel 200Amp -48VDC rectifiers
for our CO batteries with as many 125KVA delta-wye transformers in front of
them as needed for any particular site. These rectifiers are up about 98+%
efficient (better than the transformers...) but as soon as these rectifiers
became available in 480V, we switched and retrofitted everywhere except very
It is just silly to not be using 480 single or better 3 phase for very large
devices and 277V for smaller but still large devices where the single pole
breaker gives more circuits per cabinet and 277V being near previous upper
voltage limits may mean simple supply changes..
And yes, I know the Delta-Wye gives a lot of transient isolation, and a
handy "newly derived neutral" to make a really good single point grounding
system feasible and very local, but 277/480 deserves a better chance..
More information about the NANOG