Fiber only in DataCenters?

Naslund, Steve SNaslund at medline.com
Fri Dec 21 20:14:29 UTC 2012


It takes a lot of voltage to cause an arcing spark.  I would suspect
static buildup along the way and bad grounding.  Even a big facility
with a good ground should not have enough voltage differential between
grounding points to cause sparks.  Having the right size rack grounding
should give you a very low resistance to ground from any point.  The
most common problem I have seen in large facilities is multiple grounds
that are not tied together or cables that are grounded at multiple
points causing a loop current.  It is critical that everything have a
single ground, that includes racks, electrical distribution, cable tray,
etc.  Most Cat X cables are unshielded and do not have a ground
conductor so you must have equipment at the same potential at both ends
or you will get loop current for sure.

As far as voltage in Cat X cables, the real factor is the current
carrying capacity of a particular wire gage. It does not really matter
whether it is Cat 6 or a coat hanger, current capacity is a function of
cable cross section and what material it is made of.  Copper has a
specific resistance as do all other metals.  A copper cable needs to
have enough cross section to dissipate the heat generated by its
resistance.  A less conductive material requires more cross section to
dissipate the increased heat.  At extremely high voltages things become
more complex because of the skin affect that causes the power to move
through the outer parts of the cable more than the inner parts.  These
levels are not a factor in communications cables. 

The main factor for fiber over copper in data centers is all about cost.
Most servers include copper connections and fiber costs something extra.
For switches, the cost of the optics is significant.  Fiber does help
prevent damage due to surges or electrical faults but if these are a
problem in your datacenter you have bigger fish to fry.

Steven Naslund

-----Original Message-----
From: George Herbert [mailto:george.herbert at gmail.com] 
Sent: Friday, December 21, 2012 12:54 PM
To: Matthew Kaufman
Cc: nanog at nanog.org
Subject: Re: Fiber only in DataCenters?

On Thu, Dec 20, 2012 at 3:20 PM, Matthew Kaufman <matthew at matthew.at>
wrote:
> On 12/17/2012 9:22 AM, joel jaeggli wrote:
>>
>> If the facility is big enough the utility of twisted pair becomes 
>> quite limited, both due to distance and differing electrical 
>> potential, multibuilding campuses in particular make this is a
nonstarter.
>
>
> For twisted-pair Ethernet: Distance yes. Differing electrical
potential no.
> It is a balanced pair, transformer coupled at both ends. As long as AC

> common-mode pickup doesn't saturate the transformer core, it just
works.

...Up to certain limits of DC / ground differential between the ends, at
which one can cause sparks anyways.

Yes, the POTS telcos use 48V in the same or lower quality wire pairs,
and the various CatN wires should be able to take it, and the
connectors.  I'm not sure whether the sparks were from 110 or 220 V of
differential, but I saw sparks.


>> In one facility I'm in, I'm over 300 meters from each of the MMRs, 
>> with the results that the OOB for the serial console server for out 
>> equipment located out there in the MMR's being on serial over fiber 
>> transceivers connected by om4 multimode.
>
>
> RS232 serial is another story. Here the potential difference between 
> the ends is a big deal. (I've even seen burned-through PC boards from 
> what happens when pin 7 has 220 VAC flowing from one device to the 
> other) But you can just run Ethernet out to the console server and 
> plug it in next to the gear with the serial port to fix this.
>
> Matthew Kaufman

Ah, yes, those magic smokes.


--
-george william herbert
george.herbert at gmail.com





More information about the NANOG mailing list