Leap second reminder - Check your NTP

Sun Jan 1 00:57:45 UTC 2006

Last NTP spam:

I'm by no means an NTP expert, if anyone else is, please pipe up.

About 30 minutes before the leap second should have occurred, several  
of our systems reported "xntpd[13742]: time reset 0.958385 s", which  
was really strange. They moved the wrong direction, and they did it  
early. Shortly after, those systems lost ntp association and began  
drifting. About 10 minutes after midnight all have regained sync. I  
wasn't checking things that early to see why, it's possible some of  
our NTP sources started disagreeing on what the correct time was, and  
would also match what other people have reported off-list, going back  
as far as 18 hours before midnight.

Several public NTP sources are now indicating a "leap second  
alarm" (setting the leap bits to 11), which will cause most NTP  
clients to rule them out as a source. ntp-2.gw.uiuc.edu is an example:

130.126.24.44: Server dropped: Leap not in sync
server 130.126.24.44, port 123
stratum 2, precision -19, leap 11, trust 000
refid [128.174.38.133], delay 0.03357, dispersion 0.00049

According to ntpdate, its clock seems to have stopped about 5 minutes  
before midnight, and hasn't yet recovered.

Other NTP servers haven't cleared their "today is a leap second day"  
bit, which they should have by now. Some NTP implementations rule out  
servers that don't agree with what their "master" server thinks the  
leap second bits should be. My reading of the NTP spec says that at  
00:00:00 the leap bits should have been returned to zero. Attempting  
to sync from one of these servers will produce a "Next leap second  
occurs at 00:00:00.000 UTC Sun Jan 01 2006" message, but that should  
be harmless as long as they correct themselves a while before midnight.

Still others have their clocks off by a significant amount(10+  
minutes) and think they're still in sync, but since I started typing  
this email, they all have corrected themselves.

While I can't say anything broke on our network as a result of the  
leap second, a good percentage of our gear lost NTP sync or had some  
kind of NTP problem around midnight UTC. You may want to check your  
NTP status at some point, in case something drifted quite a way off  
and won't step itself back now because the difference is too great.

-- Kevin