NTP Issues Today

Tue Nov 20 20:15:55 UTC 2012

In a message written on Tue, Nov 20, 2012 at 02:28:19PM -0500, Jay Ashworth wrote:
> I'm curious, Leo, what your internal setup looks like.  Do you have an
> internal pair of masters, all slaved to those externals and one another, 
> with your machines homed to them?  Full mesh?  Or something else?

My particular internal setup is a tad weird, and so rather than
answer your question, I'm going to answer with some generalities.
The right answer of course depends a lot on how important it is
that boxes have the right time.

If you have 4 or more physical sites, I believe the right answer
is to have on the order of 8 NTP servers.  2 each in 4 sites reaches
the minimum nicely with redundancy.  These boxes can have GPS, CDMA
or other technologies if you want, but MUST peer with at least 10
stratum-1 sources outside of your network.  Of course if you have
more sites, one server in each of 8 sites is peachy.  Those on a
budget could probably get by with 4 servers total, but never less!

All "critical" devices should then be synced to the full set of
internal servers.  4 boxes minimum, 8-10 preferred.  NTP will only
use the 10 best servers in it's calculations, so there is a steep
dropoff of diminishing returns beyond 10.  For most ISP's I would
include all routers in this list.

For the  "non-critical" devices?  Well, there it gets more complex.
For most I would only configure one server, their default gateway
router.  Of course, pushing out a set of 4+ to themm if that is
easy is a great thing to do.

The interesting thing here is that no devices except for your NTP
servers should ever peer with anything outside of your network.
Why?  Let's say your NTP servers all go crazy together.  The outside
world is cut off, GPS is spoofed, the world is ending.  All that
you have left is that all of your devices are in time to each
other....so at least your logs still coorelate and such.  So having
every device under your master set of NTP servers is important.
One guy with an external peer may choose to use that, and leave the
hive mind, so to speak.

For small players, less than 4 sites, typically just use the NTP
pool servers, configuring 4 per box minimum.  If you want the same
protection I just outlined in the paragraph before, make 4 of your
servers talk to the outside world, and make everything else talk
to those.  Want to give back to the community?  Get a GPS/CDMA/Whatever
box and make it part of the NTP pool.  Want to step up your game (which
is what I do), reach out to various Stratum-1's on the net (or find
free, open ones) and peer up 8-20 of them.

> In my last big gig, it was recommended to me that I have all the machines 
> which had to speak to my DBMS NTP *to it*, and have only it connect to the
> rest of my NTP infrastructure.  It coming unstuck was of less operational
> impact than *pieces of it* going out of sync with one another...

Yep, a prime example of the scenario I described above.  Depending on
your level of network redundancy, number of NTP servers, and so on, this
is a fine solution.  With one NTP server (the DBMS) the downstream will
always use it, and stay in sync.  It's a valid and good config in many
situations.

-- 
       Leo Bicknell - bicknell at ufp.org - CCIE 3440
        PGP keys at http://www.ufp.org/~bicknell/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 826 bytes
Desc: not available
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20121120/e46d9d50/attachment.sig>