OT: Below grade fiber interconnect points

Michael Sinatra michael at rancid.berkeley.edu
Fri Nov 15 19:31:52 UTC 2013

Hi Justin and Roy:

On 11/13/13 12:05, Justin M. Streiner wrote:
> On Wed, 13 Nov 2013, Roy hockett wrote:
>> Has anyone ever used a below grade vault for housing fiber cross
>> connects?
>> We have to move a fiber interconnect facility due to the current
>> building being demolished.  If you have I would be interested in
>> talking to you.  If there are more appropriate lists, I would
>> appreciate any suggestions.
> When you say "below grade vault", do you mean something that's only
> accessible through a manhole?

In the case of CEVs, it's actually a "doghouse," an access-controlled
(cyber-lock or similar cylinder plus alarm system) entrance which also
houses the HVAC machinery.  See below.

> I haven't done this specifically, however if the vault does not have a
> controlled environment, you could be dealing with massive headaches
> related to dust/dirt contamination, moisture penetration, etc.

UC Berkeley installed 3 CEVs (Controlled Environment Vaults) below
ground on campus about 10-15 years ago.  One of them houses one of the
two main fiber penetrations to campus, including DWDM gear,
patch-panels, border routers, even packetshapers (back when those were
relevant in a large EDU environment), servers, WiFi portals, etc.  This
stuff has all been in place for at least 10 years and has worked really
well, modulo the caveats below.  Two of the vaults have 6-7 19" telco
racks, and one (the one with the big fiber entrance) also has a 23" rack
in addition to the others.


o It's hard to physically get equipment into the vault.  You'll often
need a hand-crane/winch for smaller items (servers, switches, small
routers), and a powered crane (one of those small ones that goes in the
back of a pickup truck) for larger items.  We've managed to put things
as big as a Juniper M120 in the CEV, and we have probably gotten bigger
stuff down there.  But it's not just as trivial as loading it onto a
hand-cart and wheeling it in.

o HVAC issues happen everywhere, but we've had to completely overhaul
the HVAC on one of the CEVs about 2-3 years ago (right around the time I
left UCB).  They were throwing more heat at the HVAC than was specified,
and there were a lot of custom parts for the doghouse enclosure that
weren't available anymore.  So you may find that you need to allocate
more budget to maintenance for HVAC (as well as sump pumps, CO sniffers,

o The larger of the three CEVs had an encounter with the Highway 24
Freeway overcrossing over Telegraph Avenue and had to be repaired before
installation.  (The CEVs are divided horizontally, so that there is a
top piece and a bottom piece that get installed in the ground in
sequence and then are sealed up.  In this case the top
piece--specifically the doghouse assembly--got crashed into the bridge.)
 It was repaired and has been fine, but you may have to deal with issues
like that.

o Do not wear a skirt, dress, or kilt when entering the CEV.  The
updraft of ventilation will cause issues as you climb down.  (Sounds
silly, but I have actually had to advise folks to wear long pants based
on the experiences of a manager who was touring the CEV one day.)

> I work
> in a large-campus .edu environment, so I'm some of the headaches you're
> probably trying to avoid.  Also, be aware that access to the vault could
> be an issue.  There are OSHA regs related to what sort of training and
> safety equipment someone who will be working in an underground vault
> must have.

Yes.  We have all had to go through the required OSHA training for
confined spaces and ladder safety.  The CEV system must also have a CO
sniffer, sump pump, etc.  We had a very lightweight lock-out-tag-out
system that basically involved calling the NOC when entering and exiting
the vault and also logging that information in a logbook physically at
the vault.

> I'm assuming that the fiber will be cross-connected to a new location
> prior to the building being demolished.

Been there, done that as well.  UCB had a big fiber re-splicing about 4
years ago, back when I still worked there.  In this particular case, it
went into a regular building, but the setup was similar to that of the
CEVs (it was the redundant entrance to the campus for one of the CEVs).

> Not knowing your outside plant or circumstances, would it be feasible to
> fusion-splice a new tail onto the fiber that was going to the building
> that's being demolished, or (ideally) pulling a new piece of fiber to
> the new building, so you don't have to deal with potentially dodgy splices?

So, if it's like what was done at UCB, the fiber was disconnected
(starting at 5pm, since an entire building would be down for _each_
bundle of fiber that was done) from the panel in the old building and
back-pulled to the nearest CV (not controlled environment).  It was
re-spliced in that CV and routed to the new building, reterminated and
reconnected to a new switch to bring the building back up.  (Some
bundles didn't need to be spliced, since it was a shorter path to the
new building.)  This is all doable within a CEV environment as well, and
it requires pretty much the same level of coordination and project


More information about the NANOG mailing list