Subject: [ntpwg] Timing Definitions

From: GDowd at symmetricom.com (Greg Dowd)
Date: Thu, 15 Feb 2007 14:00:22 -0800

Some standard uses for reference with ntp:

precision:  This defines the smallest measurement which can actually be
made.  It helps to define the number of significant bits in a
measurement.
resolution:  This defines the total number of bits available to express
a measurement.  It tells you nothing about the value of any of those
bits.
accuracy:  This defines how well a measurement agrees with a timescale
(e.g., UTC).

Greg Dowd
gdowd at symmetricom dot com (antispam format)
Symmetricom, Inc. 
www.symmetricom.com
"The current implementation is non-obvious and may need to be improved."

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070215/4b95419c/attachment.html

---

From: mills at udel.edu (David L. Mills)
Date: Fri, 16 Feb 2007 16:59:36 +0000

Greg,

The terms resolution and precisions are defined very narrowly in the NTP 
documents. Resolution is the reciprocal of the system clock frequency, 
usually one nanosecond with modern hardware and operating systems 
(timespec). Precision is defined as the time to read the system clock, 
currently around 500 ns to 1000 ns with modern hardware and opeating 
systems. It is important to make that distinction when calibrating the 
residual errors for accuaracy assessment. The specification document is 
very clear about this.

Dave

Greg Dowd wrote:

> ------------------------------------------------------------------------
>
> _______________________________________________
> ntpwg mailing list
> ntpwg at lists.ntp.isc.org
> https://lists.ntp.isc.org/mailman/listinfo/ntpwg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070216/042452cd/attachment.html

---

From: tglassey at earthlink.net (todd glassey)
Date: Sat, 17 Feb 2007 16:26:23 -0800

Dave - 
  ----- Original Message ----- 
  From: David L. Mills 
  Cc: ntpwg at ntp.isc.org 
  Sent: Friday, February 16, 2007 8:59 AM
  Subject: Re: [ntpwg] Timing Definitions

  Greg,

  The terms resolution and precisions are defined very narrowly in the NTP documents. Resolution is the reciprocal of the system clock frequency, usually one nanosecond with modern hardware and operating systems (timespec). 

With the new architectures however the resolution will change. The question becomes one of how to notate that and where it should be disclosed.

  Precision is defined as the time to read the system clock, currently around 500 ns to 1000 ns with modern hardware and opeating systems. It is important to make that distinction when calibrating the residual errors for accuaracy assessment. 

These also are specific to current and previously available systems but the standards will change and since there is no process for characterizing a NTP Server yet this is a reasonable commentary of Greg's.

Symmetricom and all of the HW based Time Systems providers need these data points for creating characterizations of their systems and the performance and so these additions are reasonable and warranted for supporting the commercial users of NTP.

  The specification document is very clear about this.

  Dave

  Greg Dowd wrote:

    Greg Dowd

----------------------------------------------------------------------------

------------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070217/ad68cace/attachment.html

---

From: mills at udel.edu (David L. Mills)
Date: Sun, 18 Feb 2007 18:22:44 +0000

todd,

We are on a tangent here.

The definitions I use are deliberately measurable and unambiguous. They 
are also consistent with the intricately crafted statistics budget in 
the spec, the UDel report and the book. There is nothing specific to 
present or future architecture or product. The standard will not change 
with the introduction of new architecture or product. That's the 
rigorous intent of the specification in the first place.

If you find something in the budget in error, very please honk. If you 
find something inconsistent in the name or interpretation of the error 
statistics, very please honk. Whatever honk you find must be rock hard, 
measurable and consistent with analysis.

Dave

todd glassey wrote:

> The terms resolution and precisions are defined very narrowly in the 
> NTP documents. Resolution is the reciprocal of the system clock 
> frequency, usually one nanosecond with modern hardware and operating 
> systems (timespec).
>
> With the new architectures however the resolution will change. The 
> question becomes one of how to notate that and where it should be 
> disclosed.
>
>
> Precision is defined as the time to read the system clock, currently 
> around 500 ns to 1000 ns with modern hardware and opeating systems. It 
> is important to make that distinction when calibrating the residual 
> errors for accuaracy assessment.
>
> These also are specific to current and previously available systems 
> but the standards will change and since there is no process for 
> characterizing a NTP Server yet this is a reasonable commentary of Greg's.
>
> Symmetricom and all of the HW based Time Systems providers need these 
> data points for creating characterizations of their systems and the 
> performance and so these additions are reasonable and warranted for 
> supporting the commercial users of NTP.
>

---

From: tglassey at earthlink.net (todd glassey)
Date: Sun, 18 Feb 2007 13:11:55 -0800

Dave - this was all about benchmarking NTP and setting a set of standards 
per those.

Todd

---

From: mills at udel.edu (David L. Mills)
Date: Sun, 18 Feb 2007 22:33:38 +0000

todd,

That's exactly what I had in mind, and that's why I phrased the values 
as I did in the spec. Note also jitter and wander are formally defined 
and so is delay, dispersion and distance.

Dave

todd glassey wrote:

> Dave - this was all about benchmarking NTP and setting a set of 
> standards per those.
>
> Todd
>

---

From: kurt at roeckx.be (Kurt Roeckx)
Date: Mon, 19 Feb 2007 01:31:14 +0100

> and so is delay, dispersion and distance.

I've looking for which terms are defined in the draft, and this is what
I came up with:
- resolution: Let p be the number of significant bits in the second
  fraction. The clock resolution is defined 2^(-p), in seconds.
- precision: The clock precision is defined as the running time to read the
  system clock, in seconds. Note that the precision defined in this way can
  be larger or smaller than the resolution. The term rho, representing the
  precision used in the protocol, is the larger of the two.
- offset: The offset theta represents the maximum-likelihood time offset of
  the server clock relative to the system clock.
- delay: The delay delta represents the roundtrip delay between the client
  and server.
- dispersion: The dispersion epsilon represents the maximum error inherent
  in the measurement. It increases at a rate equal to the maximum disciplined
  system clock frequency tolerance PHI, typically 15 PPM.
- jitter: The jitter psi, defined as the root-mean-square (RMS) average of
  the most recent time offset differences, represents the nominal error in
  estimating theta.
- wander: The RMS average of past frequency offset differences
  represents the oscillator frequency stability or frequency wander
  OMEGA
- distance: The synchronization distance LAMBDA = EPSILON + DELTA / 2
  represents the maximum error due all causes.

So, what I couldn't find was one for accuracy.  The distance is
probably closest to that.

But I want to give an example, so I know I understand things right.  If
my kernel returns the time in microseconds, but only updates it say at
100 Hz.  I think my resolution is 10 ms, but would set p to 6, so I get
.0156 seconds.  If I can read the clock say 1000 times a second, I think 
my precision would be 1 ms. But since I'd read the same value 10 times,
I'd have to set rho to the resolution.

Kurt

---

From: mills at udel.edu (David L. Mills)
Date: Mon, 19 Feb 2007 05:32:44 +0000

Kurt,

You raise an important point. There is no such thing as accuracy in the 
product, only a statistical estimation of the resolution, precision and 
other statistics. There is no specific accuracy statement possible in 
this thing. It's like saying, how pretty is your girlfriend.

Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070219/dc803404/attachment.html

---

From: seaman at noao.edu (Rob Seaman)
Date: Mon, 19 Feb 2007 07:35:40 -0700

On Feb 18, 2007, at 10:32 PM, David L. Mills wrote:

> There is no such thing as accuracy in the product, only a  
> statistical estimation of the resolution, precision and other  
> statistics. There is no specific accuracy statement possible in  

(One is tempted to point out that the way to answer the latter  
question is to ask one's wife.)

Isn't the same issue as traceability?  Perhaps convolved with the non- 
parametric nature of the clock discipline?  The whole point of  
timekeeping is to tie a remote clock to a master time standard.  This  
is distinct, say, from identifying physical limits on the accuracy of  
absolute intervals returned by an egg timer.

In general, unless a trusted measurement of whatever quantity is  
available, an estimate of accuracy is impossible.  The mental picture  
usually used to convey the difference between accuracy and precision  
is target practice.  Tight clustering of bullet holes is precision.   
Having the average pattern spread coincide with the center of the  
target is accuracy.

The two things missing from that last statement are first, an  
indication of the repeatability and reliability of the shooting/ 
measurement.  What did the same marksman do last time (to provide  
confidence for next time)?

And secondly, the point in contention with this thread is how do you  
know where the center of the target lies?  With target shooting the  
answer is that the target is much larger than the bullet in both  
absolute size and solid angle.  With timekeeping, we demand that our  
fiducial systems of time and our portable/remote estimates of same  
look and behave like the same things.  Our bullets and our targets  
are the same size.

There would be (relatively) little difficulty in using a nanosecond  
standard timekeeper to measure the accuracy of a tenth-second laptop  
clock - at least, little difficulty if you remove the vagaries of the  
intervening network in favor of something other than NTP...

Rob Seaman
National Optical Astronomy Observatory

-------------- next part --------------
A non-text attachment was scrubbed...
Name: accuracy_precise.gif
Type: image/gif
Size: 5481 bytes
Desc: not available
Url : https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070219/ed0157f9/attachment.gif 
-------------- next part --------------

(http://www.globalsecurity.org/military/library/policy/navy/misc/ 
image434.gif)

---

From: kurt at roeckx.be (Kurt Roeckx)
Date: Mon, 19 Feb 2007 19:18:54 +0100

> Having the average pattern spread coincide with the center of the  

The maximum error in a measurement is the sum of the error caused by
accuracy and precision.  What we do is calculate the maximum error (and
estimate the error).  From this number you can't split it up in which
part is caused by the accuracy and which by the precision error.

If we could somehow calculate the accuracy error, and the resolution of
our clock is smaller than that, we could reduce it to something in the
order of our resolution.

Anyway, depending on the application, you're either interrested in the
maximum total error or in the precision.  I can't think of something
were you're interrested in the accuracy.

Kurt

---

From: mills at udel.edu (David L. Mills)
Date: Tue, 20 Feb 2007 00:56:05 +0000

Kurt,

A few years ago I was at a retreat in Dagstuhl, southern Genmany, 
jousting NTP with serious computer science theory folks. I was the only 
engineer in the room; the rest were theory cats and didn't know the 
difference betweena phase-lock loop and piston return spring. That was 
quite an education for both of us.

According to the theory cats, The dispersion increase (maximum error) 
would have to be at the frequency tolerance limit, assumed 500 PPM. 
However, this is unrealistic, so I settled on 15 PPM as the disciplined 
clock dispersion increase. From a rigorous computer science perspective, 
the NTP architecture is incorrect. My point being that the maximum 
assumed error depends on that assumption. Somebody else might make a 
different assumption, and yes, the increase can be tinkered.

As for extimated error, the error budget has been intricately crafted 
with painful attention to detail. A blow by blow breakdown about the 
various contributions is in the spec and in more detail in the report. 
The book contains even more mathematical detail and justification.

The interpretation of resolution and precision is very important, as the 
resolution depends on the tick interval in some machines, the system 
timer in others and the TSC in still others. The precision statistic is 
somewhat arbitrary, but would satisfy the theory cats, as the value of 
the system clock is indetermenant during the system call.

It is incorrect to describe the error as the sum of accuracy and 
precision. The expected error is the combined offset measured from the 
cluster algorithm survivors. This is the best estimate relative to the 
server(s), but does not take into account the server offset relative to 
its sources. The clock jitter is the estimated error of making that 
measurement and does take into account all the servers down to the 
reference clock(s), a most conservative statistic.

Stand next to Westminister Abbey and set your watch to Big Ben. Big Ben 
clockkeepers keep the clock in tune by listening to the BBC and putting 
coinage on the pendulum. Presumably, the "accuracy" of Big Ben depends 
on the residual frequency error and the interval between coinage 
updates. You set your watch to Big Ben and inherit its characteristics 
and some of your own. How "accurate" is your watch?

Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070220/e65a30fd/attachment.html

---

From: kurt at roeckx.be (Kurt Roeckx)
Date: Wed, 21 Feb 2007 00:14:18 +0100

Hi Dave,

> and some of your own. How "accurate" is your watch?

Since I have no idea about what the accuracy or precision is of Big
Ben, it's going to be rather hard to say anything about it.  And I
think that's the point we're both trying to make.

Anyway, there are 2 ways to synchronize your clock to Big Ben.  One
is looking at it and reading the time.  This is as far as I know with
a resolution of 1 minute, so a precision of atleast 1 minute.  But the
clock might be set with an accuracy in the order of a few seconds.  The
other is listening to it's bells which is going to have a higher
precision but a different accuracy.

To say anything useful about any of those 2 ways, you need to do
statistics by comparing it to a clock that has a small enough error.

If I'm going to set my watch by any of those 2 methods, what matters is
which way has the smallest total error.  The accuracy of my watch is
going to depend on that, and not how accurate the clock is or how
precise I can "read" it.

You can go and reduce the error caused by the precision by taking
more samples, but you can't compensate for the accuracy error.

Anyway, the definition of precision and accuracy I use might be a
little bit different than what you're used to when talking about NTP.
If you take some samples and compare it to a reference, the accuracy
would be the average difference between the samples and the reference,
the precision would be the standard error.

What you use in NTP for the precision is the highest of the resolution
and the time it takes to read the clock.  This is a reasonable estimate
for how precise you can measure the local system clock.  It's not a good
estimate for how precise a remote ntp daemon can come and read your
system clock since it doesn't only depend on your own system now.  It's
this "remote" precision that is used to synchronize the clock.  And I
think you try to estimate that by the jitter.

Kurt

---

From: seaman at noao.edu (Rob Seaman)
Date: Tue, 20 Feb 2007 18:01:52 -0700

>> and some of your own. How "accurate" is your watch?

This example is also given in Dr. Mills' excellent book.

> think that's the point we're both trying to make.

I don't think so.  Some protocol could be implemented to convey  
information regarding Ben's current load of coinage and about how far  
it leads or lags the BBC.  It isn't ignorance of these metadata that  
causes trouble in estimating accuracy, it is the lack of a  
fundamental standard.

> Anyway, there are 2 ways to synchronize your clock to Big Ben.

I'm sure there are many more than two.  This reminds me of the  
barometer:

	http://www.snopes.com/college/exam/barometer.asp

I can even add one more.  I've got a nifty new MacBook with the  
accelerometer chip.  The SeisMac app comes with a library that can  
poll three axis values at a couple hundred Hz.  Placing a MacBook in  
close enough proximity to the tower's clock would allow ticks to be  
detected.  Ben is actually the name of the bell, and one might well  
be able to count the striking of the hours from outside the  
building.  Anybody in London want to try?

> which way has the smallest total error.

What matters is whether you have a handy fundamental reference to  
consult - see the appended message from the leapsecs list, for  
example.  (Any followups about leap seconds should be directed that  
way - not for the faint of heart.)  The utility of NTP is precisely  
in conveying an estimate of a fundamental standard to a remote  
locale.  At NOAO we rely on NTP on Kitt Peak in Arizona and Cerro  
Tololo in Chile.  Imagine if the weights and measures folks had to  
recalibrate a meter, a liter or a kilo from 5000 miles away -  
continually, 24/7.  The nice thing about UTC (or other flavor of  
Universal Time) is that we're all sitting on top of the fundamental  
reference.

To return to the point in question.  The current draft looks good.   
Wrap it up with a pretty bow.  It's about time you guys started  
working on ntpv5...

Rob Seaman
------

  	From: 	  seaman at noao.edu
	Subject: 	Re: [LEAPSECS] how to reset a clock
	Date: 	January 4, 2007 8:15:41 AM MST
	To: 	  LEAPSECS at ROM.USNO.NAVY.MIL

Peter Bunclark wrote:

> Indeed isn't this Rob's ship's chronometer?

Actually, I think it was Mr. Harrison's.  (And Steve Allen has been  
basing his arguments more recently on this distinction.)  This  
healthy debate between astronomical time and clock time has happened  
before.  The answer is the same as before - both types of time are  
needed.  (Some things never change.)  I'm sure Pete is more familiar  
with this story than I am, but others may not be.

Harrison attempted to build a perfect clock to win the Longitude  
Prize.  Folks who haven't read Sobel's book should do so - my  
classmates at Villanova and I learned the story from an Augustinian  
priest who appeared old enough to have known Harrison personnally.   
Harrison's first glorious shipboard clock failed to take the prize  
due to a lack of compensation for centrifugal effects on a sailing  
vessel that must tack when sailing against the wind (or must wear  
through an even larger angle, bringing the wind across its stern).   
Compensation was needed for relativistic effects, if Newtonian rather  
than Einstein.  (Some things never change.)

Harrison invented or improved a variety of familiar mechanical  
doodads like the roller bearing and bimetallic temperature  
compensation.  He likely could have succeeded in solving this  
particular problem, but there would always have been another physical  
improvement needed.  (Some things never change.)  Each improvement  
would have made the clock more complicated and eventually too fragile  
to possibly work on a constantly moving platform buffeted and often  
bathed by the salty sea.

He created a second clock and was working on a third round of  
improvements when the idea we're discussing first occurred to him.   
He had been using a pocket watch as a mechanism to transfer time from  
stationary standard clocks (many built by himself) to his portable  
prototypes.  He would reset the clock in one place and physically  
carry it to where the time was needed.  If a roundtrip correction  
were needed, presumably he would note the time on either end and  
halve the difference.  This is the "standard synchrony" or  
"conventionality of simultaneity" of special relativity - familiar to  
anyone who has looked under the hood of NTP.  (Some things never  
change.)

What Harrison recognized was that he didn't need to build a perfect  
clock - he merely needed to quantify and log the error inherent in  
the clock.  By replacing a large and finicky "better" clock, with a  
small and robust, but more even-tempered, one, the rate of the clock  
could be regularized and its random and systematic errors could be  
minimized.  That the rate of the clock was now guaranteed not to  
match the rate of the spinning Earth was no longer a bug, but a  
feature.  By carefully calibrating the clock rate before leaving on a  
voyage, and checking it against astronomical observations throughout  
the voyage, it was possible to compute the mean solar time at the  
home port.  (Some things never change.)  Comparison with the local  
time, measured by sextant, then recovered the longitude directly.

And, of course, a ship would not carry a single clock, but two or  
more.  Friendly ships meeting at sea would also exchange clock  
readings - creating the first ensemble time scale.  (Some things  
never change.)

Thus was the chronometer born - and thus did Britannia rule the waves.

The point is that time isn't just an unending count of seconds - it  
is the epoch of when the count was zero.  That epoch often has  
significance in some periodic natural phenomena, usually related to  
Earth orientation.

Rob

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ntp.isc.org/pipermail/ntpwg/attachments/20070220/a103ca0e/attachment-0001.html