Subject: [ntpwg] Further to the timestamping issue

From: "David L. Mills" <[email-masked]>
Date: Sat, 14 Jun 2008 21:31:50 +0000

Guys,

Not to bring up a dead horse, but I don't think everybody was completely 
happy about the timestamp capture issue; at least, I have few comments 
about the white paper Stuart Venters and I prepared. However, and mainly 
to clear the air with a practical, worked-out scheme, I have updated the 
NTP On-Wire Protocols briefing at 
http://www.eecis.udel.edu/~mills/database/brief/onwire/onwire.ppt and 
white paper http://www.eecis.udel.edu/~mills/onwire.html to fix a few 
types, cleanup the presentation and add an interleaved broadcast mode. 
For yucks I included a short description of the NASA/JPL Proximity-I 
timestamping scheme used on Mars orbiters and landers. Cool.

For reference, the symmetric/broadcast interleaved protocol is backwards 
campatible with the present protocol, resists lost, duplicate and bogus 
packets and automatically adapts to basic and interleaved protocol 
modes. It has been thoroughly tested in simulation for both normal and 
error conditions.

If I can figure a way to capture TSC/PCC contents in the Ethernet 
driver, this puppy will bark. I've peeked at the FreeBSD Ethernet device 
driver for the Digital Tulip (if_de.c) and found it vaguely readable. 
I'm not a device driver guru by any means, , although I feel like an 
elephant in a china shop. The idea is to append the TSC to the end of 
the receive and transmit buffer when the frame is complete. Crude, but 
this is an expereiment, not a specification proposal. I have a couple of 
machines with Broadcom NICs that the if_de.c driver can handle.

Comments and/or guffaws humbly accepted.

Dave
_______________________________________________
ntpwg mailing list
[email-masked]
https://lists.ntp.org/mailman/listinfo/ntpwg

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 14 Jun 2008 21:57:13 +0000

In message <[email-masked]>, "David L. Mills" writes:

>happy about the timestamp capture issue; [...]

If we are going to roll a packet format revision, would it be prudent
to consider all identified issues with the NTPv4 format:

* Add a y2036 disabiguation field
  This is cheaper than increasing the width of all timestamps.
  One bit will go a long way, two would be enough for all of us.

* Add (optional) DUT1 transmission field

* Change/Define contents of Root Delay to be useful/meaningful
  Practically all stratum 1 servers are uncalibrated/undefined.

* Change/Define Root Dispersion to be useful/meaningful
  While statistically correct, the number is of no use and often
  deeply misleading.

* Fix reference identifier
  So it can handle IPv6 correctly.

* Add support for optionally transmitting timezone info to clients.
  For the chosen timezone transmit ZoneName, UTC_offset, DST start/end.

* Fix leapsecond handling.
  Add timestamp of next scheduled leapsecond, so that clients can
  handle the event correctly, even if they are offline at the time.

And that's just the ones I can remember at 11pm a saturday night...

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
ntpwg mailing list
[email-masked]
https://lists.ntp.org/mailman/listinfo/ntpwg

---

From: "David L. Mills" <[email-masked]>
Date: Mon, 16 Jun 2008 03:56:47 +0000

P-H,

I said nothing about new headers, only a reuse of the existing ones in 
interleaved modes, as detailed in the documents cited. However, you 
raise some other issues, with comments below. Inline comments are hard 
for me because for eyesight reasons I have to reformat in fixed-width 
and MIME fights me every step of the way.

Dave

> One bit will go a long way, two would be enough for all of us.

We have had this discussion before. If you consider NTP a mechanism to 
set the clock in any era, even across eras, and agree to set it within 
68 years in the current era, no changes are necessary. In other words, 
if in 2038 you set the system time between 1970 and 2006 (give or take 
Unix), the clock will be correctly set. I did this experiment with 
current NTP and Solaris and it worked fine.

But, if you expect NTP to set the clock in Octobrer, 1582 starting from 
the 21st century, you would need the NTP era number. That could be 
carried in an extension field.

> * Add (optional) DUT1 transmission field

Again, an extension field. Like leapsecond wargings, you will need to 
define a mitigation procedure if more than one value was reported from 
local and/or remote sources.  You also need to specify how to get this 
intormation to potential applications. For this, I suggest the kernel 
interface ntp_gettime() as modified.

> While statistically correct, the number is of no use and often

The root statistics are carefull designed to extend to the reference 
clock source itself, optionally including its sources. Usually, the 
radio is considered the source, so the root delay is zero. The 
dispersion is referenced to the last update received over the radio 
channel, not the last update from the radio protocol, but not all 
drivers respect that.  In the case of the WWV driver, the delay budget 
extends to the Boulder antenna. For space use the intent is to include 
the coding delay and the lightime to the source, usually on Earth.  The 
jitter already does include the jitter of the radio measurements 
themselves, as calculated by the median filter.  For space use, this 
would include the jitter of the space link and coding/interleaving 
jitter. I submit the current semantics are correct, although maybe not 
as clear in the documentation. If you found this confusing, you should 
have spoken up before.

> So it can handle IPv6 correctly.

This would require a real and backwards-incompatible change. We can 
discuss this at another time.

> For the chosen timezone transmit ZoneName, UTC_offset, DST start/end.

Personally, I  think this is out of scope of NTP, but I might not be the 
one to decide should the "next version" take a jink.

> handle the event correctly, even if they are offline at the time.

You need to be more explicit about "fix leapsecond handlin"g. The 
current scheme might not be what you assume. See the current 
documentation. You forgot to mention the TAI. As it stands now, the 
next-leap, leaptable-expire and TAI are already available in the kerne 
and retrievable via ntp_gettime() on all systems but Linuxl. These data 
are currently available only if Autokey is running and should be a 
separable, extendable feature.

> And that's just the ones I can remember at 11pm a saturday night...

At 0354Z on Monday morning, keep those cards and letters coming.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 16 Jun 2008 12:16:48 +0000

In message <[email-masked]>, "David L. Mills" writes:

>I submit the current semantics are correct, although maybe not 

I did not say they were not correct, I said they were of no use.

And as a result of that, they are more likely than not to be
correct on the wild network since people don't bother setting
them right for their stratum 1 servers.

Some of this originates with the software, for instance most DCF77
refclocks do not use the same definition of root delay as your
quoted for WWVB.

>You need to be more explicit about "fix leapsecond handlin"g.

The current leapsecond handling gives a very short warning about
an impending leapsecond, despite the fact that we know six months
in advance that it will happen.

A device which is "off the net" during the short interval around
the leapsecond, will not know to do the right thing.

This is a particular problem with DCF77 stratum1 servers which 
only get 1 hour leap-second warning in PTB's transmissions.

We should add a field that can contain the time of the next
scheduled leapsecond:

	"Next leapsecond is at [32bit seconds]"

So we can set it at the time when we receive The Bulletin from Mr.
Gambis.

---

From: "David L. Mills" <[email-masked]>
Date: Mon, 16 Jun 2008 14:33:58 +0000

> refclocks do not use the same definition of root delay as your

I mentioned WWV, not WWVB.With WWV I can accurately characterize the 
propagation path up to the antenna. Most drivers don't have knowledge of 
the transmission path, but very little is gained by doing that anyway. 
The intent of the delay/dispersion/jitter semantics is to provide a 
maximum error statistic and expected error statistic. Device driver 
authors should read the specification and understand the error budget. 
If other drivers don't follow the intended semantics, that's not the 
fault of the semantics.

> So we can set it at the time when we receive The Bulletin from Mr.

Pleas, please, read the documentation. Leapsecond handling is not as you 
assume. Warnings can be months in advance if armed from the leapseconds 
file. The leapseconds values include the epoch of the next/last leap, 
the epoch of expiry and the TAI at the next leap and up to a month if 
from leap bits. We don't need a field for the next leap; we already have 
that. And, once the epoch has been determined, the client can go off-net 
and still leap the leap.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 16 Jun 2008 15:27:04 +0000

>If other drivers don't follow the intended semantics, that's not the 

I'm not saying it is, but I'm advocating adding some metrics with
semantics people can understand and use.

Even if they were correctly implemented, the root delay and root
dispersion is of no use to NTP users.

>The leapseconds values include the epoch of the next/last leap, 

I fail to fully appreciate how you compressed that into the
two bit field of the NTPv4 protocol.

Or are you talking about some undocumented extension field ?

---

From: "TSG" <[email-masked]>
Date: Mon, 16 Jun 2008 08:59:20 -0800

----- Original Message ----- 
From: "Poul-Henning Kamp" <[email-masked]>
To: "David L. Mills" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, June 16, 2008 7:27 AM
Subject: Re: [ntpwg] [ntp:hackers] Further to the timestamping issue

> semantics people can understand and use.

hear hear!

---

From: Rob Seaman <[email-masked]>
Date: Mon, 16 Jun 2008 09:53:37 -0700

> in advance that it will happen.

First, thanks to Poul-Henning for raising these issues.

I would be as concerned as PHK, if his assertion is true.  Dave says  
it is not.  Which is it?  Does NTP practice differ from NTP theory?

> the leapsecond, will not know to do the right thing.

A device can be unreachable for either a short or long interval.  One  
obvious issue is that the local clock will drift, so the leap second  
will only be synchronized locally - but that is true of everything  
done with the clock during such an interval.

The other case is if the device is down, not simply out.  In this  
case, the leap second should (perhaps) be omitted.  Basically it turns  
into a pragma to inform the clock's initialization method.

In either case, the question is whether the leap second schedule has  
been persisted locally in advance.  This seems no different than any  
other appointment calendar application.  If an appointment is missed,  
it is skipped - no rescheduling for leap seconds.  On the other hand,  
if a clock ticks in the woods and there is nobody to hear - the alarm  
still rings.

David L. Mills wrote:

> Warnings can be months in advance if armed from the leapseconds

All factions of the leap second debate support lengthening the six  
month scheduling lookahead.  This would require no change to any  
standard.  Is NTP ready to accept leap second schedule updates years  
in advance - and still realize reliable leap second handling?  If not,  
what needs to change?

There has also been discussion on LEAPSECS about publishing a rolling  
schedule that looks forward through several leap seconds.  It appears  
that the state of the art for Earth Orientation modeling may soon  
allow sufficiently accurate prediction for a lookahead of several  
years.  This would change the NTP "arming" paradigm into something  
much more clearly recognizable as a distributed appointment schedule.

Which is to say that NTP now assumes that there are periods of time  
when a leap second has been announced, and periods when no leap second  
has been announced.  In the future - and with no need to change ITU  
460-4 - it may be the case that (multiple) leap seconds have  
perpetually been announced to the world.  (The soonest ITU 460-4 might  
change now appears to be 2019.)

I especially like the distinction in http://www.cis.udel.edu/~mills/leap.html 
  between how UTC reckons leap seconds and how NTP deals with them.   
The former would remain unchanged - the latter may have to adapt (and  
perhaps to NTP's betterment).

> two bit field of the NTPv4 protocol.

See ntpEntStatusLeapSecond and ntpEntStatusLeapSecDirection.  How  
these values are loaded is an implementation detail, e.g., Autokey.   
There certainly is no requirement that every packet contain the full  
schedule.

Rob Seaman
National Optical Astronomy Observatory

---

From: "David L. Mills" <[email-masked]>
Date: Tue, 17 Jun 2008 01:46:23 +0000

P-H,

The the maximum error statistic is defined as half the root delay plus 
the root distpersion. The root disperision is the error due to the 
maximum credible frequency error of the primary reference source 
multipled by the time since last synchronized by that source. Additional 
minor statistics are added as defined in the specificiation. It may be 
your opinion that a maximum error statistic is not userful, and you are 
welcome to that opinion.

Yes, I did juju magic and managed to cram three 32-bit words into two 
leap bits. But, you have to look further in the documentation to find 
out how I did that. You might start by looking for the leapseconds file 
and, should the mystery about how other machines find out about it, you 
might enjoy the autokey specification. The leapseconds extension field 
shouldn't require autokey. Gotta fix that someday.

Dave

---

From: "David L. Mills" <[email-masked]>
Date: Tue, 17 Jun 2008 02:31:55 +0000

Bob,

I don't understand what you mean by "current behavior"; the currennt 
behavior is documented on the ntpd page in the online documentation. 
Earlier this year I reworked the code to handle appointment errors as 
you describe. However, you might be using an older version. There was 
some discussion on this list at that time; you might find it in the 
archives.

Briefly put, a leap appointment is made from the NIST leapseconds file 
or imported from a server using an autokey extension field. When the 
clock is set, a counter is set as the seconds to the leap, if there is 
one. The NIST file expires and must be retrieved from time to time and 
appointments remade if necessary. In principle, you could make an 
appointment in July to take effect in December and then go offline until 
then.

The WWV and WWVB leap warnings apply at the end of the current month. If 
an appointment has not been made from either the NIST file or autokey 
and a majority of peer leap warning bits are set, a leap appointment is 
made for the end of the month. If less than 23 hours remain to the leap, 
the system leap bits are set. If the clock is stepped or the system 
restarted of less than a majority of peer leap warning bits are ser, the 
appointment is cancelled.

The only requirent is that the clock must be updated during the day 
before the leap in order to arm the kernel. Presumably, should you go 
offline, either the local clock driver or orphan mode is configured to 
take care of that. Addtional details are in the ntp_proto.c source file.

Dave

---

From: "David L. Mills" <[email-masked]>
Date: Tue, 17 Jun 2008 02:59:35 +0000

P-H,

On reflect, I don't know whether your discain for the NTP statistics 
budget is due to a conclusion that the maximum error and expected error 
statistics are not useful and/or confusing or that my calculation of 
them is faulty. It would be useful to find out what your replacements 
might be.

Dave

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 17 Jun 2008 11:24:54 +0000

>them is faulty. It would be useful to find out what your replacements 

Stratum 1 admins don't pay any attention to calibrating them correctly.

And since they are uncalibrated, their values have no or little
relation to the actual metrics.

Thus: Useless in practice.

But I don't know if they are by definition useless in theory: The
latest definitions I have been able to find are:

   Root Delay (rootdelay): Total round trip delay to the reference
   clock, in NTP short format.

   Root Dispersion (rootdisp): Total dispersion to the reference clock,
   in NTP short format.

But it is not obvious what is meant by "reference clock".

Is that the piece of hardware connected to the S1 server, or is it
the far-away atoms which governs the output of that piece of hardware ?

If they only measure back only to the S1 server, then they are
indeed useless in theory:  You would need to query the S1 server
to see where it lives and what refclock it uses, so you can compensate
for the diurnal changes in propagation delay in case it is a VLF
receiver, and for the jitter in case they used a cheap serial
alarm-clock type receiver. 

If they refer back to the distant atoms, then 99% of all VLF based
S1 servers on the net are badly uncalibrated (up to tens of
milliseconds) and most of the rest merely trivially uncalibrated
(tens of microseconds).

In that case, they are possibly not useless in theory, but because
the majority of servers are uncalibrated, we cannot know for sure.

Since there is no way to get all the servers out there to shape
up, any remedy would entail new fields with usable data, for
servers which are calibrated, and a pass-note from uncalibrated
servers.

Poul-Henning

---

From: "David L. Mills" <[email-masked]>
Date: Tue, 17 Jun 2008 15:08:58 +0000

Poul,

It might not be clear from the documentation, as most folks might 
consider it an overly pedantic issue, but there is a serious formal 
model here. I consider three disciplined oscillators with respect to a 
primary server. First is the remote source, such as the WWV antenna. 
Second is the local reference oscillator in the GPS receiver or in the 
WWV case the audio codec. Third is the system clock disciplined by NTP. 
The device driver interface design is intended to allow contributions 
from the first two sources to be budgeted.

For example, the refclockproc structure includes reference timestamp, 
offset, dispersion and jitter intended to be inherited from the 
reference clock driver and its source. The delay is bugeted from the 
fudgetime values. The driver can budget these items either with respect 
to the receiver oscillator or the other end of the link, as does the WWV 
driver. The intent is to allow the driver to be as pedantic as 
justified. The intent is that a client in the high stratum desert know 
the delay and dispersion to the primary source, whether it be in 
Boulder, Mainflingen or a GPS receiver. For most drivers the distinction 
might not be significant, but the capability is there.

Dave

---

From: "TSG" <[email-masked]>
Date: Wed, 18 Jun 2008 15:48:47 -0800

Dave

----- Original Message ----- 
From: "David L. Mills" <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Sent: Tuesday, June 17, 2008 7:08 AM
Subject: Re: [ntp:hackers] Further to the timestamping issue

> from the first two sources to be budgeted.

meaning that while the budgeted sources are weighted in how they are applied 
the OS Clock is not and this is the core problem with a Software Only Clock 
system.

I would propose that NTP doesnt really need to be tied to a SW only clock 
and in fact the NTP Server Package and not the NTP protocol is what ties it 
to a SW based clock model.

That said, it seems totally reasonable that in instances where there is an 
embedded HW based timekeeping system like a 1588MC or similar, that the NTP 
calls should be able to point to it rather than only getting time off the 
local host-based OS Clock through the local NTP Service Daemon.

Todd

---

From: Rob Seaman <[email-masked]>
Date: Wed, 18 Jun 2008 16:47:47 -0700

TSG wrote:

> it seems totally reasonable that in instances where there is an  
> embedded HW based timekeeping system like a 1588MC or similar, that  
> the NTP calls should be able to point to it rather than only getting  
> time off the local host-based OS Clock through the local NTP Service  
> Daemon.

You appear to be talking about something entirely different than what  
Dave and PHK were discussing.  However...

Why should an external clock, locally attached, be believed more than  
an external clock attached to a remote server?  Either NTP is a way  
for two clocks to keep better time than either alone - or we should  
stop using it.

Rob Seaman
NOAO

---

From: Rob Seaman <[email-masked]>
Date: Wed, 18 Jun 2008 17:14:18 -0700

Hi Dave,

> behavior is documented on the ntpd page in the online documentation.

I think there was some confusion here with my words and a quote from  
PHK.  My fundamental point is that UTC permits much more flexible leap  
second scheduling than has been demonstrated to date.  NTP might have  
to adapt to this in the future.

> some discussion on this list at that time; you might find it in the

I found a message as you describe from June of 2007.  We're running  
modern code - ignoring legacy hosts :-)

> appointment in July to take effect in December and then go offline  

Ok.  With the multi-leap-second lookahead scenario I was discussing,  
this would have to be robust against several intervening intercalations.

> appointment is cancelled.

Sounds as reasonable as any other heuristic.

> before the leap in order to arm the kernel.

Ok..  So arming during the preceding day, and the possibility of  
maintaining several future leap second appointments are separate  
actions.  Arming in this sense is reasonable.  Arming six months in  
advance would not be.

Thanks for the clarification.

Rob

---

From: "M. Warner Losh" <[email-masked]>
Date: Wed, 18 Jun 2008 18:49:00 -0600 (MDT)

picking a random email to reply to...

The problems that I've had with leapseconds:

(1) ntpd only advertises them a day in advance, which means brief
    outages can miss them.
(2) IRIG signals them only an hour in advance, which is too slow to
    drive ntpd from.
(3) There's no way to get the current offset easily from ntp servers
    since so few of them have the crypto setup to get the
    leapsecond.txt file.  This is especially true in "sensitive"
    networks with no connectivity to the internet.
(4) Machines off for a long time require a long time to get what the
    current UTC/TAI offset is.  GPS is good and can give the current
    offset, but there's no mechanism to get the intermediate values.
    Not totally critical, but customers notice when the historical
    leap seconds we made up in this case are wrong....  This is the
    'cold spare' problem...

These issues can be worked around, but they are all a pita.

Warner

---

From: "TSG" <[email-masked]>
Date: Thu, 19 Jun 2008 07:15:02 -0800

Rob I wanted to respond to this comment of yours.

----- Original Message ----- 
From: "Rob Seaman" <[email-masked]>
To: <[email-masked]>
Sent: Wednesday, June 18, 2008 3:47 PM
Subject: Re: [ntpwg] Further to the timestamping issue

> Dave and PHK were discussing.  However...

No not really. But as to Leap Second management my response to you below 
should allow for that as well.

> an external clock attached to a remote server?

It shouldnt unless it is a master calibration resource like a properly 
calibrated 5071A for instance. The question is whether the remote clock is 
in fact a clock and not a reference oscillator.

For the same reason it would be believed remotely. Because there is 
calibration regimen which document's either remote clocks' operations.

> Either NTP is a way

My point was whether certain REFCLOCK operations should be allowed to be the 
actual NTP Server's Time Source and not the TOD meaning that NTP becomes an 
interface to that HW system and a standardized protocol for interrogating 
that time-keeper device.

In my model the NTP Listener would query and sidestep the TOD register read 
through the REFCLOCK interface for devices which were more than reference 
oscillator's - i.e. that had a TOD calibrated into them from a CAL LAB.

This would allow for really stupid things like multiple sources of Time to 
be used inside the same systems (something probably foreign to you Rob but a 
really powerful idea for creating ensemble based timeservice models in a 
single system).

The idea that I was using a 5071A as an absolute source of time info means I 
want to query the reference device directly , not have some shared-use OS 
query it for me. Its why IEEE1588 is so powerful in an NTP world. In the 
best of all possible worlds, I want to be able to have three or more voices 
voting in an ensemble and a weighting model for their accuracy, jitter, and 
drift.

That means that NTP which becomes a single voice is a problem the way it is 
today.  What it needs is a way to abstract its timekeeping from the HOST OS 
TOD management so that the OS can be truely a client of the time-keeping 
process and not a client/facilitator of that process.

---

From: "TSG" <[email-masked]>
Date: Thu, 19 Jun 2008 07:04:36 -0800

----- Original Message ----- 
From: "Rob Seaman" <[email-masked]>
To: "David L. Mills" <[email-masked]>
Cc: "NTP Working Group" <[email-masked]>
Sent: Wednesday, June 18, 2008 4:14 PM
Subject: Re: [ntpwg] Further to the timestamping issue

> to adapt to this in the future.

Not if its used in proper auditing of IT operations. Leap Seconds are a 
nightmare for logging systems because they require special handling and 
integration across an entire entitie's operations.

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 08:38:21 -0700

On Jun 19, 2008, at 8:04 AM, TSG wrote:

>> My fundamental point is that UTC permits much more flexible leap
> Not if its used in proper auditing of IT operations. Leap Seconds  
> are a nightmare for logging systems because they require special  
> handling and integration across an entire entitie's operations.

I don't discount auditing requirements, although it isn't clear that  
NTP's requirements are the same as your company's.  That the Earth  
orbits the Sun and has a large nearby Moon are a bit more fundamental  
than either.  We can disagree about the implications of all sorts of  
requirements.  Basing a discussion on requirements is generally more  
productive, however, than jumping directly into each other's nightmares.

NTP already provides special handling and integration for the case of  
leap seconds.  That being the case, it certainly would be appropriate  
to update the system should the underlying UTC leap second  
implementation ever change.  This is not imminent, but seems  
ultimately more likely than the current fad for advocating the  
elimination of leap seconds in a fruitless attempt to abandon mean  
solar time.

In any event, NTP either supports UTC or it doesn't.  This is also a  
requirement, whatever complications that causes elsewhere.  Required  
complexity is another name for "design".

In short, UTC *already* permits multiple leap seconds to be announced  
in advance. with an arbitrarily longer horizon than six months.  NTP  
systems *already* fail to implement the full complexity implicit in UTC.

Rob Seaman
NOAO

---

From: "M. Warner Losh" <[email-masked]>
Date: Thu, 19 Jun 2008 09:55:17 -0600 (MDT)

In message: <[email-masked]>
            Rob Seaman <[email-masked]> writes:
: On Jun 19, 2008, at 8:04 AM, TSG wrote:
: 
: >> My fundamental point is that UTC permits much more flexible leap
: >> second scheduling than has been demonstrated to date.  NTP might have
: >> to adapt to this in the future.
: >
: > Not if its used in proper auditing of IT operations. Leap Seconds  
: > are a nightmare for logging systems because they require special  
: > handling and integration across an entire entitie's operations.
: 
: I don't discount auditing requirements, although it isn't clear that  
: NTP's requirements are the same as your company's.  That the Earth  
: orbits the Sun and has a large nearby Moon are a bit more fundamental  
: than either.  We can disagree about the implications of all sorts of  
: requirements.  Basing a discussion on requirements is generally more  
: productive, however, than jumping directly into each other's nightmares.

That only matters if you think that having civil time that's closely
tied to the mean solar time is important.  After all, there's very few
people that live at the meridian where the localtime is defined.
Today, my clock is just about an hour off from mean solar time due to
daylight savings time.  It just doesn't matter enough in people's
lives, and causes other problems, such as the logging issues above.

I know Rod loves leap seconds.  I doubt that will ever change.  But
they are a big pain in the ass that nobody ever bothers to get right
(due to the cost, ignorance if the issues, etc), but everybody wants
to be right.  It would be easier to publish DUT1 go on the drift and
move national time zones every hundred years or so.  Lord knows the
politicians move them around a lot more often than that.

Rod is pedantically correct too when he says that UTC can do leap
seconds more often than twice a year.  The large number of things that
break if that were to happen shows just how fragile the leap second
scheme is.  ntpd is but one example (since it had a filter to only
announce leap seconds at the end of June and December), and ntpd is
written by people very clueful about leap seconds.

Anyway, enough ranting.  Leap seconds are evil and must die.  Others
will no doubt disagree.

Warner

---

From: <[email-masked]>
Date: Thu, 19 Jun 2008 17:14:31 +0100

I just take the view that with all of the processing power and expertise
we have nowadays, if a company can't build something that  can cope with
something as simple as leap seconds, I don't want to buy anything else
from them. Especially if it has anything to do with safety equipment!

	Tony Flavin
	Core Networks Engineering Manager

	e-mail: [email-masked]
	tel: +44(0) 1473 609570
	mobile: +44(0) 7801 759596
	fax: +44(0) 1908 862752

	Orion 5,  pp11 , Adastral Park, Martlesham, Ipswich IP5 3RE, UK

-----Original Message-----
From: ntpwg-bounces+anthony.flavin=[email-masked]
[mailto:ntpwg-bounces+anthony.flavin=[email-masked]] On Behalf Of
M. Warner Losh
Sent: 19 June 2008 16:55
To: [email-masked]
Cc: [email-masked]
Subject: Re: [ntpwg] Further to the timestamping issue

: >> second scheduling than has been demonstrated to date.  NTP might
have

: than either.  We can disagree about the implications of all sorts of  
: requirements.  Basing a discussion on requirements is generally more  
: productive, however, than jumping directly into each other's
nightmares.

That only matters if you think that having civil time that's closely
tied to the mean solar time is important.  After all, there's very few
people that live at the meridian where the localtime is defined. Today,
my clock is just about an hour off from mean solar time due to daylight
savings time.  It just doesn't matter enough in people's lives, and
causes other problems, such as the logging issues above.

I know Rod loves leap seconds.  I doubt that will ever change.  But they
are a big pain in the ass that nobody ever bothers to get right (due to
the cost, ignorance if the issues, etc), but everybody wants to be
right.  It would be easier to publish DUT1 go on the drift and move
national time zones every hundred years or so.  Lord knows the
politicians move them around a lot more often than that.

Warner
_______________________________________________
ntpwg mailing list
[email-masked] https://lists.ntp.org/mailman/listinfo/ntpwg
_______________________________________________
ntpwg mailing list
[email-masked]
https://lists.ntp.org/mailman/listinfo/ntpwg

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 09:44:39 -0700

M. Warner Losh wrote:

> tied to the mean solar time is important.

I'll keep this short since LEAPSECS provides the proper place for such  
discussions.  There are any number of related threads over the years:

	http://six.pairlist.net/pipermail/leapsecs

> After all, there's very few people that live at the meridian where  
> the localtime is defined.
> Today, my clock is just about an hour off from mean solar time due  
> to daylight savings time.

This confuses periodic and secular issues.  A static or seasonal  
offset is one thing.  A quadratically accelerating drift is another.   
The world is riddled with dependencies on the length of the solar day.

> I know Rod loves leap seconds.

Rather I care about mean solar time.  Leap seconds are just a means to  
an end.

> UTC can do leap seconds more often than twice a year.

NTP already handles this case, but it is a different issue than I was  
discussing.

Rob

---

From: "M. Warner Losh" <[email-masked]>
Date: Thu, 19 Jun 2008 10:53:57 -0600 (MDT)

In message: <[email-masked]>
            Rob Seaman <[email-masked]> writes:
: M. Warner Losh wrote:
: 
: > That only matters if you think that having civil time that's closely  
: > tied to the mean solar time is important.
: 
: I'll keep this short since LEAPSECS provides the proper place for such  
: discussions.  There are any number of related threads over the years:
: 
: 	http://six.pairlist.net/pipermail/leapsecs
: 
: > After all, there's very few people that live at the meridian where  
: > the localtime is defined.
: > Today, my clock is just about an hour off from mean solar time due  
: > to daylight savings time.
: 
: This confuses periodic and secular issues.  A static or seasonal  
: offset is one thing.  A quadratically accelerating drift is another.   
: The world is riddled with dependencies on the length of the solar day.

A quadratically accelerating drift doesn't fit leap seconds well
either...  A few hundred years and we'll need leap seconds way too
often to be practicable without major changes...  Also, I'm not
confusing periodic and secular issues.  To keep noon constant, you
just move the local offset by an hour when it has time has drifted
enough...  It just expands the tolerance...

: > I know Rod loves leap seconds.
: 
: Rather I care about mean solar time.  Leap seconds are just a means to  
: an end.
: 
: > UTC can do leap seconds more often than twice a year.
: 
: NTP already handles this case, but it is a different issue than I was  
: discussing.

The ntp protocol does.  The last ntpd I looked at specifically
filtered out all leap second indicators that weren't done on the last
day of June or December.

Warner

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 09:55:34 -0700

TSG wrote:

> I want to query the reference device directly , not have some shared- 
> use OS query it for me.

?  Your application would still be layered on the OS.

---

From: "TSG" <[email-masked]>
Date: Thu, 19 Jun 2008 09:59:02 -0800

Rob (in the best spirits - my response is below :-)

Todd

----- Original Message ----- 
From: "Rob Seaman" <[email-masked]>
To: "NTP Working Group" <[email-masked]>
Sent: Thursday, June 19, 2008 7:38 AM
Subject: Re: [ntpwg] Further to the timestamping issue

> NTP's requirements are the same as your company's.

Ahahhah - NTPs requirement's are for a trusted system to deploy time-of-day 
calibration services to distributed computer's.

The question here is whether those are what NTP.ISC.ORG is up to or it has 
some other agenda...

> That the Earth
> leap seconds.

Sort of...

> That being the case, it certainly would be appropriate
> implementation ever change.

So then you are saying that it is more important to add a new security risk 
that to live within how it operates now? Yeah right... :-)

> This is not imminent,

My point exactly.

>but seems

Unfortunately it doesn't talk about the precision or performance of NTP in a 
loaded system. Or what the interrupt processesing latency or overhead would 
be in the bigger picture. So we have no idea how a NTP Server is supposed to 
respond, whether it can sit there and cache all the requests to serve them 
all at once or otherwise. Or whether it can be paged into la-la land while 
other higher priority processes use those same compute resources which NTP 
needs to keep its TOD accurate.

>
> In any event, NTP either supports UTC or it doesn't.

Ah yes - but it really doesnt support UTC anyway. Currently NTP supports a 
"TAI+" type scale with some external clock source - otherwise the timescale 
NTP supports in that of its own  ("TAI+" means Last UTC Setting + TAI Ticks 
since then). The real question is whether NTP should autocorrect and add 
leap-seconds  automatically or not and the answer is NO WAY.

Leap Second's are a nightmare for correlating actual TOD through. The 
mapping of that to UTC happens once a month when the BIPM does the 
Circular-T thingee and proclaims UTC to be what it is for that month.

Besides since its computed after the fact, UTC is not a time-scale that 
ANYONE can keep a beating clock to. They can keep it to TAI + Last(UTC 
Fixing) and that is it. Poof - end of statement.

> This is also a

The problem is that NTP(SW) [NTP's Software Only Timescale] is not a 
reliable time scale nor can it be relied on in an evidentiary standard for 
anything IMHO since you cannot specifiy what else that compuer was doing at 
the time it was supposed to be answering the NTP Query.

> in advance.

Do you have any idea of the cost of implementing a log management practice 
that will address multiple Leap Second's being added???

Considering also that Leap Second's are never added such that there would be 
more than one per month added, the accidental adding of a leap second 
(assuming NTP automatically tracks them) could cause billions or trillions 
of dollars of damage and skewing to our commerce systems that support their 
automatic induction.

> with an arbitrarily longer horizon than six months.  NTP

And for good reasons too.

---

From: "TSG" <[email-masked]>
Date: Thu, 19 Jun 2008 10:00:41 -0800

Depends
----- Original Message ----- 
From: "Rob Seaman" <[email-masked]>
To: "NTP Working Group" <[email-masked]>
Sent: Thursday, June 19, 2008 8:55 AM
Subject: Re: [ntpwg] Further to the timestamping issue

> ?  Your application would still be layered on the OS.

Sure but the issue is whether the TOD is maintained in that OS or in some 
device specifically built for that purpose. Like a 5071A for instance.

Todd

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 10:11:58 -0700

Hi Warner,

I'm not going to buy your arguments here any more than in LEAPSECS,  
where they belong :-)

Interested members of the peanut gallery are directed to the archives.

>> NTP already handles this case

> filtered out all leap second indicators that weren't done on the  
> last day of June or December.

As with your message yesterday, these are things to fix, not work  
around.

NTP implements UTC.  UTC currently requires leap seconds.  Legal leap  
second scheduling policies are broader than currently supported by  
NTP.  This is a good thing since it permits the longer range schedule  
desired by many.  Lengthening the schedule is where our efforts should  
be directed.

Rob

---

From: "David L. Mills" <[email-masked]>
Date: Thu, 19 Jun 2008 17:20:50 +0000

Warner,

> (1) ntpd only advertises them a day in advance, which means brief

Not quite true. When from the leapseconds file or autokey, the warnings 
are indefinate. True, the server leap bits are set only on the day, so 
if no other source was available than a reference clock with radio bits, 
the server would not show those bits other than the day of the leap.

> (2) IRIG signals them only an hour in advance, which is too slow to

No IRIG format I am aware of has provisions for leap warning.

> networks with no connectivity to the internet.

The only thing necessary to get the TAI offset is the kernel 
ntp_gettime() syscall, which Linux does not have at present. The autokey 
is necessary only to download the leapsecond values from a server/peer. 
That's a crock and should be changed. The leapseconds table itself is 
loaded even without autokey.

> leap seconds we made up in this case are wrong.... This is the

By :intermediate values" I assume you mean the historic leapseconds 
table, which is available via FTP from NIST. I submit this is not an 
issue with NTP; the issue should be owned by the timezone folks, who 
should also show leapsecond warning and TAI offset available from IERS. 
Another possiblity is SNMP.

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 10:28:42 -0700

TSG wrote:

> Do you have any idea of the cost of implementing a log management  
> practice that will address multiple Leap Second's being added???

No, and neither do you.

You are also jousting with a windmill.  Nothing would stop Gambis from  
announcing a leap second for December 2011 at the same time as a leap  
second for June 2009.  (Dates chosen randomly.)  Each would be handled  
in sequence whether or not NTP is aware of both at the same time.   
This is precisely a feature of UTC that has been requested.  The  
question is how best would NTP implement it.

> Considering also that Leap Second's are never added such that there  
> would be more than one per month added,

Nice argument against leap hours :-)

> the accidental adding of a leap second (assuming NTP automatically  
> tracks them) could cause billions or trillions of dollars of damage

Somebody who places auditing above all other requirements might want  
to actually characterize the cost/schedule/performance benefits and  
risks of the various alternatives :-)

Rob

---

From: Steve Allen <[email-masked]>
Date: Thu, 19 Jun 2008 11:11:28 -0700

On Thu 2008-06-19T10:28:42 -0700, Rob Seaman hath writ:
> Nothing would stop Gambis from
> second for June 2009.

Except for the very strong "sic semper faciebamus" which lingers from
the totally haphazard way that actions by the BIH, IAU, and CCIR in
the 1960s and 1970s led to the current practices.  With two of those
agencies no longer in existence any change is hard to effect.

The world would be better off if the broadcast precision time scale
(and "broadcast" now includes internet, POSIX time_t as well as radio)
did not have leap seconds.  The 2003 colloquium on UTC in Torino
admitted as much.  Leap seconds are very suited for humans, and not
suited for machines.

What the colloquium did not point out was that the POSIX specs for
zoneinfo already require enough complexity to handle the leap seconds,
and the mechanisms for updating the zoneinfo files are well tested.
All that's needed is to rename the time scale used in broadcasts,
NTP, and POSIX time_t.

NTP already contains the necessary complexity, if not the desired
features.  All of the missing bits would be much easier to solve
by putting leap seconds in zoneinfo than by messing with NTP.

This discussion should go to LEAPSECS and leave NTPWG.

--
Steve Allen                 <[email-masked]>                WGS-84 (GPS)
UCO/Lick Observatory        Natural Sciences II, Room 165    Lat  +36.99855
University of California    Voice: +1 831 459 3046           Lng -122.06015
Santa Cruz, CA 95064        http://www.ucolick.org/~sla/     Hgt +250 m
_______________________________________________
ntpwg mailing list
[email-masked]
https://lists.ntp.org/mailman/listinfo/ntpwg

---

From: "David L. Mills" <[email-masked]>
Date: Thu, 19 Jun 2008 18:16:41 +0000

Rob,

I don't understand your comment. The IERS can surely announce any future 
leap epoch, but from the current draft of the relevant ITU-T and NIST 
recommendation, there are no provisions with public dissemination means 
to announce other than in June or December. The WWV/H/B and ACTS leap 
warning indicators apply at the end of the current month, which is how 
the leap warning bits are interpreted in NTP. From past experience, 
server leap warning bits have had less than perfect reliability and 
represent a possible vulnerability. This is why the upstratum servers 
vote on the leap warning bits and are  reluctant to set their own leap 
warning bits other than on the day of the leap.

Dave

---

From: "TSG" <[email-masked]>
Date: Thu, 19 Jun 2008 11:26:38 -0800

All you need is a leap-second listener protocol to make this work. EIRS 
would support that I bet too.

Todd

---

From: "David L. Mills" <[email-masked]>
Date: Thu, 19 Jun 2008 18:36:07 +0000

Rob,

You imply a burning desire to predict leap events possibly years in 
advance. From the IERS bulletins I gather announcing leap seconds is 
somewhat of a black art and the announcements are only a few months in 
advance, depending on the DUT1 crossover. Also, the IERS can announce 
multiple bulletins even now should they choose; however, the time 
dissemination community has no means at present to announce more than 
one at a time. In fact, it is possible to announce more than one 
occasion in the NIST leapseconds file. Should that possibility arise, I 
would be delighted to have NTP properly respond to them.

Dave

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 11:48:34 -0700

> This discussion should go to LEAPSECS and leave NTPWG.

I'm copying it over there.  Please remove ntpwg from further replies.

> POSIX specs for zoneinfo already require enough complexity to handle  
> the leap seconds, and the mechanisms for updating the zoneinfo files  
> are well tested.  All that's needed is to rename the time scale used  
> in broadcasts, NTP, and POSIX time_t.

Just to be clear, the realization of UTC would then occur locally via  
zoneinfo mediated leap seconds.  This is quite different from the  
notion of letting the timezones slosh around.

> All of the missing bits would be much easier to solve by putting  
> leap seconds in zoneinfo than by messing with NTP.

If we could escape from the artificial crisis generated from the  
relentless ITU shenanigans, we would have time to focus on  
characterizing the problem before suggesting solutions.

Rob

---

From: "David L. Mills" <[email-masked]>
Date: Thu, 19 Jun 2008 19:13:52 +0000

Todd,

As of this morning the development branch can accept hardware 
timestamps. However, the kernel clock apparatus is still necessary in 
order to interpolate time betwen timestamp capture. This limits 
potential accuracy to a couple of microseconds due to the kernel clock 
swish and sway. It would in principle be possible to implement the 
kernel clock as an external clock device of higher quality and we once 
did that with a purpose-built bus peripheral. P-H has done that using a 
rubidium oscillator. Provisions for that are in the precision kernel code.

Provision for multiple primary standard oscillators requires a means to 
develop a synthetic timescale, which is usually done after the fact. 
Primary standard oscillators provide only a frequency; they have to be 
calibrated to establish the seconds epoch. I calibrate mine by 
physically transporting it to USNO, where a technician brings a BNC 
cable to the truck and pushes the calibrate button. A candidate NTP 
timescale including multiple standard oscillators and intended for 
possible implementation and deployment is described in my book. It is 
based on the ARIMA scheme used at NIST.

Dave

TSG wrote:

> Dave
>
> ----- Original Message ----- From: "David L. Mills" <[email-masked]>
> meaning that while the budgeted sources are weighted in how they are 
> applied the OS Clock is not and this is the core problem with a 
> Software Only Clock system.
>
> I would propose that NTP doesnt really need to be tied to a SW only 
> clock and in fact the NTP Server Package and not the NTP protocol is 
> what ties it to a SW based clock model.
>
> That said, it seems totally reasonable that in instances where there 
> is an embedded HW based timekeeping system like a 1588MC or similar, 
> that the NTP calls should be able to point to it rather than only 
> getting time off the local host-based OS Clock through the local NTP 
> Service Daemon.
>
> Todd
>
>>
>

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 11:56:12 -0700

Hi Dave,

> I don't understand your comment. The IERS can surely announce any  
> future leap epoch,

That was my comment.

> The WWV/H/B and ACTS leap warning indicators apply at the end of the  
> current month, which is how

> represent a possible vulnerability. This is why the upstratum  
> servers vote on the leap warning bits and are  reluctant to set  
> their own leap warning bits other than on the day of the leap.

As I said in some message, this heuristic seems reasonable to me - but  
moving this responsibility to zoneinfo as Steve Allen suggests might  
reveal even better trade-offs.

Rob

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 19 Jun 2008 16:33:24 -0400

Dear Dave;

Leapseconds are driven by torques at the core mantle boundary, which  
we cannot see directly
and basically have information about only through UT1 and LOD  
measurements. In practice, leap seconds cannot
be predicted much more than 6 months to a year in advance because of  
that.

AND, keep in mind that in the "jerk" seen around 1905 would have meant  
several _negative_ leap seconds in
one year, which would I am sure be a fun experience if repeated today.

Regards
Marshall

---

From: Rob Seaman <[email-masked]>
Date: Thu, 19 Jun 2008 16:17:35 -0700

Replies to leapsecs.

On Jun 19, 2008, at 1:33 PM, Marshall Eubanks wrote:

> measurements. In practice, leap seconds cannot be predicted much  
> more than 6 months to a year in advance because of that.

Modeling of Earth rotation has become exquisitely good.  (Don't  
neglect information from seismology.)  Discussions on LEAPSECS suggest  
that useful predictions are possible several years in advance.  The  
horizon can be lengthened further by loosening the amplitude of DUT1 a  
skosh.  A consensus appears likely around a set of parameters allowing  
for about a decade lookahead.

Note that the related science of helioseismology has progressed to the  
point that sunspots can be "seen" on the far side of the Sun.  Hidden  
engines are studied quite productively throughout the sciences.

> AND, keep in mind that in the "jerk" seen around 1905 would have  
> meant several _negative_ leap seconds in one year, which would I am  
> sure be a fun experience if repeated today.

We don't have leap seconds due to current variations, but due to  
integrating the already accumulated LOD offset of a few milliseconds.   
As the tidal offset grows quadratically, it has become virtually  
impossible for a negative leap second to occur, since such a jerk as  
you describe would have to persist long enough to overcome the  
accumulated temporal moment arm.  The effective LOD zero point is in  
the nineteenth century, not the 1970's.

Rob Seaman
NOAO