What to do when your ISP off-shores tech support

Todd Vierling tv at pobox.com
Fri Dec 26 19:14:50 UTC 2008

On Fri, Dec 26, 2008 at 2:01 AM, Mark Foster <blakjak at blakjak.net> wrote:
> Aside from the typical Degree or Diploma that tertiary outfits offer,
> there's not a lot of good ways to 'break in' to the Network and Systems
> Operations communities other than good ol experience,
> working-from-the-bottom-up.

I'm working in management of software engineering now, and in my
experience, the only worthwhile candidates for hiring -- who have not
gone through the self-teaching and self-experimentation phases that
mirror working at a helpdesk on a small scale -- have progressed
through exactly this chain.  They have developed the necessary
instincts to know when a bug could become a serious problem at 2 a.m.
on a Sunday, instincts that are an absolute prerequisite to working on
software intended to be used 24/7.

In software development, new college grads can be OK for
non-operationally-facing applications, but they tend to have high
ideals, and just haven't "had their hearts broken" by business
contradictions or operational emergencies yet.  On the opposite side
of the spectrum, those who have gone through only regimented software
processes between school and the present tend not to be aware of
operational impacts at all, as they've been shielded from that aspect
all along.

> So as you move your Tier 1's offshore, you cut off the channel by which
> people can gain experience and move on up the chain...

We're seeing this more and more as time goes on.  What's worse is that
offshoring of software development was becoming just as rampant,
resulting in the double-whammy of "engineers" not knowing the
consequences of their actions, and operations caught unaware when
those consequences manifest as critical problems.

Many businesses have at least partially learned from this mistake the
Hard Way, by losing customers when there was no one capable of fixing
a critical problem within 24 or even 72 hours.  Alas, this hasn't been
heeded by all of the market yet.

All of the above is solely my opinion, and definitely represents an
experience-diluted version of my personal ideals.  While I generally
agree from a business perspective that offshoring of operations can be
a lucrative cost-cutting measure, the key problem in most such
arrangements is that the operations and systems
(hardware/software/networks as applicable) are not *all* offshored at
once.  When these bits do not exist in relatively close proximity to
each other, communications between their responsible folks grinds to a

-- Todd Vierling <tv at duh.org> <tv at pobox.com> <todd at vierling.name>

More information about the NANOG mailing list