Subject: [ntpwg] Proposed REFID changes

From: Harlan Stenn <[email-masked]>
Date: Sun, 19 Jul 2015 22:11:45 -0700

Folks,

I thought I sent these before, but I'm not seeing it in my archives.

The attached change goes to an additional way to flag a refid at S2+ as
being an IPv6 refid.

I have another change in my queue, one where an NTP server that is
offering "smeared" leap second corrections would use a refid of
254.a.b.c, where a.b.c is 2 bits of signed integer and 22 bits of
unsigned fractional seconds.  This value is the amount of smear that has
been applied to the timestamps in the packet.  But I see I haven't
finished that writeup yet.

Newer versions of ntpd (and people) can also use this to detect when a
time server is sending them smeared time.

I believe these changes are safe because initial octets of 254 and 255
are not permitted as source addresses in an IPv4 address.

I probably won't be able to attend this meeting - it was scheduled for
0300-0500 my time, and while I'm usually falling asleep around 0300, if
I stay up until 0500 it's light outside and at that point I have a lot
of trouble falling asleep.

-- 
Harlan Stenn <[email-masked]>
http://networktimefoundation.org - be a member!
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 20 Jul 2015 09:53:21 +0200

Harlan,

> finished that writeup yet.

Hm. As far as I can see this is not only in your queue but in the 
current -stable code. :-(

I really don't like this idea.

> are not permitted as source addresses in an IPv4 address.

Is there any precaution in the code that the first byte of the hash 
generated from an IPv6 address can never be 254?

If this could be the case then clients could assume a smearing server 
just because the hash from an upstream server's IPv6 address is 254.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 20 Jul 2015 10:18:54 +0200

Folks,

as far as I know, fortunately this list of well-known refid strings is 
neither complete, nor mandatory:

      +------+----------------------------------------------------------+
      | ID   | Clock Source                                             |
      +------+----------------------------------------------------------+
      | GOES | Geosynchronous Orbit Environment Satellite               |
      | GPS  | Global Position System                                   |
      | GAL  | Galileo Positioning System                               |
      | PPS  | Generic pulse-per-second                                 |
      | IRIG | Inter-Range Instrumentation Group                        |
      | WWVB | LF Radio WWVB Ft. Collins, CO 60 kHz                     |
      | DCF  | LF Radio DCF77 Mainflingen, DE 77.5 kHz                  |
      | HBG  | LF Radio HBG Prangins, HB 75 kHz                         |
      | MSF  | LF Radio MSF Anthorn, UK 60 kHz                          |
      | JJY  | LF Radio JJY Fukushima, JP 40 kHz, Saga, JP 60 kHz       |
      | LORC | MF Radio LORAN C station, 100 kHz                        |
      | TDF  | MF Radio Allouis, FR 162 kHz                             |
      | CHU  | HF Radio CHU Ottawa, Ontario                             |
      | WWV  | HF Radio WWV Ft. Collins, CO                             |
      | WWVH | HF Radio WWVH Kauai, HI                                  |
      | NIST | NIST telephone modem                                     |
      | ACTS | NIST telephone modem                                     |
      | USNO | USNO telephone modem                                     |
      | PTB  | European telephone modem                                 |
      +------+----------------------------------------------------------+

The list above mentions "DCF", but the parse driver from the NTP package 
sets the refid to "DCFp" if the DCF77 receiver decodes the phase 
modulation, which provides better accuracy, and pure AM receivers with 
reduced accuracy may get the refid "DCFa". Others really use just "DCF".

Meinberg has been suggesting to their customers to fudge the refid for 
some refclocks to something like "DCFi" or "GPSi" for DCF77 or GPS PCI 
cards ("internal") which are accessed through the parse driver.

In current versions of Meinberg's driver software the time from the PCI 
card can alternatively be fed to ntpd via ntpd's shared memory segments, 
which provides better accuracy than the approach with the parse driver. 
In this case we suggest to our customers that they use refids like 
"GPSs" to indicate the "shared memory" approach.

As far as I can see refid "PTP" is not only only used for "Euopean 
telephone modem", but is also returned by the PTB's public NTP servers.

Finally, the list mentions "GPS" for GPS receivers, and "GAL" for 
Galileo receivers. How about GLONASS and Beidou receivers?

Lots of current versions of "GPS" modules can receive signals from 
different satellite systems at the same time, e.g. combinations of GPS, 
Glonass, Beidou, and Galileo. So maybe a refid "GNSS" could be more 
appropriate for these types of receivers.

Beside this, please keep in mind that ntpd uses a number of special 
refid strings internally. A quick grep for sys_refid yields e.g. "STEP", 
"DOWN", "INIT", and "LOOP".

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Harlan Stenn <[email-masked]>
Date: Mon, 20 Jul 2015 01:40:46 -0700

Hi Martin,

> I really don't like this idea.

Why?

There are complaints that the current IPv6 hash is difficult because
folks can't tell that it's a hash.  Going to 255.a.b.c fixes this.

Once this is done, it's no great leap to use 254.a.b.c to identify a
server that is smearing a leap second.

I don't buy the argument that this will cause timing loops.  If there is
no timing loop when there isn't a leap second, why would there suddenly
be a timing loop during the leap second if we used 254.a.b.c to identify
that we're offering smeared time

Also, smeared time is only offered to client associations.  One would
expect two machines that might have a timing loop to use a peer
association, not a server association.

With the current code there is no way to tell if a server is offering
smeared time.  With this proposal it becomes obvious.

> generated from an IPv6 address can never be 254?

Not in the old code.  In the new code, yes.

But also today, ntpd itself doesn't try and detect if it's getting
regular time or smeared time.  The 254.a.b.c is currently used more for
humans than ntp.  This can change in the future, so if one gets into the
situation where there are 2 smearing servers and 2 normal servers, we'd
be able to "reverse-engineer" the smear amount and come up with the
correct time, and have a better chance of finding a majority clique.

> just because the hash from an upstream server's IPv6 address is 254.

For this to happen the clients would have to know what that first byte
of the refid meant, and right now no versions of ntpd have that knowledge.

---

From: Florian Weimer <[email-masked]>
Date: Mon, 20 Jul 2015 11:26:57 +0200

> finished that writeup yet.

That's a bit gross.  Defining optional, transitive extensions (similar
to what BGP has for path attributes) and using this for this purpose
seems much more useful and cleaner.

Won't some clients react in subtle ways to a refid that changes in every
response?

>      | HBG  | LF Radio HBG Prangins, HB 75 kHz                         |

This should mention “CH”, not “HB”.  And it seems this time signal has
been decommissioned.

-- 
Florian Weimer / Red Hat Product Security
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Harlan Stenn <[email-masked]>
Date: Mon, 20 Jul 2015 02:39:59 -0700

Hi Florian,

> seems much more useful and cleaner.

That will be something to look at for NTPv5.  I'm looking for things
that will work with NTPv4, as deployed.

> Won't some clients react in subtle ways to a refid that changes in every

I can't speak for all clients.  I know that ntpd will use the refid to
detect a 1-degree timing loop.  The only time this will potentially
'break' is during the time a smeared leap second is being offered, and
I'm not sure what use-cases there are where a timing loop would occur
when A makes a *client* request to B, and B is also making *client*
requests to A.

At that time, a well-functioning client will check the refid to see if
it has an IPv6 address configured for the other side, generate the hash,
grab the first 4 bytes, and if it doesn't match the refid then it tries
again using 255 as the first byte.  If that matches it's a loop.

If the hash is such that the first byte is already 254 and the rest of
the bytes match the hash, then we would assume a timing loop and ignore
the packet.

If the remote machine really is giving us smeared time, then on the next
poll the refid will be different, so the false-positive loop detection
will not be an issue.  Note there is a 1 in 16,777,216 chance of this
false-positive happening.

Again, there are currently no versions of ntpd out there that look to
see if the first byte of the refid is 254 and use that knowledge to
"know" that the other side is sending smeared time.  The 254.a.b.c refid
is currently only used by "humans".  We have time to decide what ntpd
should do if it detects that a CLIENT response is arriving with q 254
refid prefix.  A client response is the *only* response that can come
with a leap smear.

> This should mention “CH”, not “HB”.  And it seems this time signal has

I don't think the above came from me.

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 20 Jul 2015 12:10:17 +0200

>> I really don't like this idea.

We have discussed this widely when I implemented the smearing support. 
IMO the only *proper* way to transport the smear offset to clients would 
be to add it in an extension field.

> folks can't tell that it's a hash.  Going to 255.a.b.c fixes this.

Has this also already been implemented in the current -stable code?

> server that is smearing a leap second.

This is what I currently get on a client of a smearing server, while 
smearing is in effect, using both ntpq and ntpd from current -stable 
4.2.8p3 on the client:

:/# date -u; ntpq -p
Tue Jun 30 23:13:57 UTC 2015
      remote       refid   st t when poll reach   delay   offset  jitter
=======================================================================
*lt-martin.py.me ...+6.    1 u   63   64   37    0.398   29.063  18.257

:/# date -u; ntpq -p
Tue Jun 30 23:14:03 UTC 2015
      remote       refid   st t when poll reach   delay   offset  jitter
=======================================================================
*lt-martin.py.me ...)0.    1 u    2   64   77    0.337   28.919  15.389

You see the refid is only displayed as garbage which quickly changes 
over time as the smear offset send by the server varies. The calls above 
were just run 6 seconds after each other.

> that we're offering smeared time

I'm not worried about timing loops, but about confusion of the users 
when they see things like the above.

If the server should fudge the refid while smearing is in progress then 
at least the status monitoring programs from the same package version 
should know how to handle this and display something useful instead of 
garbage.

It was just a quick shot to pull this in, and in the current state of 
the code is is not useful at all, IMO.

> smeared time.  With this proposal it becomes obvious.

Yes. See above. :-(

> Not in the old code.  In the new code, yes.

Is 4.2.8p3 the old code or the new code?

> correct time, and have a better chance of finding a majority clique.

Yes, and re-generate a leap second warning flag for the kernel locally, 
so that the kernel can handle a leap second in the usual way.

What if such a cool, future client gets its time from an older server 
which just sends 254.x.y.z as an IPv6 hash. The cool new client then 
assumes the server is smearing a leap second, and tries to undo the 
smearing locally?

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: "Ulrich Windl" <[email-masked]>
Date: Mon, 20 Jul 2015 13:51:08 +0200

Hi!

In addition to MArtin's comments, maybe pojnt out the fact that the ID can be
changed by the user, so deriving any expectations from the ID by the user may
be misleading...

So maybe this may be "recommendations for naming".

Regards,
Ulrich

>>> Martin Burnicki <[email-masked]> schrieb am 20.07.2015 um
10:18 in

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 20 Jul 2015 13:53:26 +0200

On Sun, Jul 19, 2015 at 10:11:45PM -0700, Harlan Stenn wrote:
>    Above stratum 1 (secondary servers and clients): this is the
>    reference identifier of the server and can be used to detect timing
>    loops.  If using the IPv4 address family, the identifier is the four-
>    octet IPv4 address.  If using the IPv6 address family, it is the
>    first four octets of the MD5 hash of the IPv6 address.  Note that,
>    when using the IPv6 address family on an NTPv4 server with a NTPv3
>    client, the Reference Identifier field appears to be a random value
>    and a timing loop might not be detected.
> 
> I recommend changing:
> 
>    If using the IPv6 address family, it is the
>    first four octets of the MD5 hash of the IPv6 address.
> 
> to:
> 
>    If using the IPv6 address family, it is either the
>    first four octets of the MD5 hash of the IPv6 address, or a
>    first octet of 255 followed by the second, third, and fourth
>    octets of the MD5 hash of the IPv6 address.

-1 from me. (FWIW)

This is an incompatible change. It would break the detection of timing
loops on hosts using the old definition of refid, which seems to be
the primary purpose of the refid field.

-- 
Miroslav Lichvar

---

From: "Ulrich Windl" <[email-masked]>
Date: Mon, 20 Jul 2015 13:57:02 +0200

Hi!

I cannot resist the imagination of Dave Mills' face chaging color three times per seconds when reading about smeared leap seconds ;-)
I think Google does it (= smear leap seconds, not change the colors). Microsoft Windows smears the time in longer intervals all the year (AFAIK). Fortunately Windows does not provide NTP services. Now what if a user wants a "always smear time" mode?
That's not NTP any more IMHO.

Regards,
Ulrich

>>> Harlan Stenn <[email-masked]> schrieb am 20.07.2015 um 10:40 in Nachricht

---

From: brian utterback <[email-masked]>
Date: Mon, 20 Jul 2015 09:42:05 -0400

I think we should work backwards. We have time before the next leap
second to get it right, as long as we don't drop it. So, assuming we are
working NTPv5, what would we do? Then once we have an idea of the right
thing to do in an ideal world, how close can we get in NTPv4? And once
we know that, what can we do about legacy software? I think that this is
the proper order. Working the other way means we might just make the
problem harder down the road.

-- 
Oracle <http://www.oracle.com>
Brian Utterback | Principal Software Engineer
Phone: +1 6038973049 <tel:+1%206038973049>
Oracle Systems/RPE Solaris Network
1 Oracle Dr. | Nashua, NH 03062
------------------------------------------------------------------------
All working systems eventually start to exhibit their own agenda.
------------------------------------------------------------------------
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
developing practices and products that help protect the environment
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Steve Allen <[email-masked]>
Date: Mon, 20 Jul 2015 07:02:04 -0700

> That's not NTP any more IMHO.

If the REFID changes to identify leap smear, should it also change to
identify an NTP server which is serving UT1 instead of UTC?

http://www.nist.gov/pml/div688/grp40/ut1_ntp_description.cfm

--
Steve Allen                 <[email-masked]>               WGS-84 (GPS)
UCO/Lick Observatory--ISB   Natural Sciences II, Room 165   Lat  +36.99855
1156 High Street            Voice: +1 831 459 3046          Lng -122.06015
Santa Cruz, CA 95064        http://www.ucolick.org/~sla/    Hgt +250 m
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: [email-masked]
Date: Mon, 20 Jul 2015 18:52:43 +0200

RFC 5905 introduced extension field to add new features to NTP. The 
smeared leap second is a new feature. I therefore support Martins 
suggestion to transport the smear by means of an extension field.

Dieter

"ntpwg" <ntpwg-bounces+dieter.sibold=[email-masked]> wrote on 
20.07.2015 12:10:17:

> From: Martin Burnicki <[email-masked]>
> To: Harlan Stenn <[email-masked]>, [email-masked]
> Date: 20.07.2015 12:39
> Subject: Re: [ntpwg] Proposed REFID changes
> Sent by: "ntpwg" <ntpwg-bounces+dieter.sibold=[email-masked]>
> 
> Harlan Stenn wrote:
> > Hi Martin,
> >

---

From: Harlan Stenn <[email-masked]>
Date: Mon, 20 Jul 2015 11:03:27 -0700

Hi Martin,

> be to add it in an extension field.

That's *a* proper way, but I don't think it is *the* only proper way.

I also think it's insufficient.  How can a human know that a server
offering smeared time where the amount of smear is only visible in an
extension field know that time from that server is being smeared?

Also, I think we *must* add the smear to the root dispersion.

This also means it might be better to change the smear from being fully
applied at the moment of the actual end of the leap second insertion to
being applied *half* applied at that time, with the remaining half being
applied over the next period.  That will limit the error to at most a
half a second.

> Has this also already been implemented in the current -stable code?

No.  I have a bit more coding to do still.

>      remote       refid   st t when poll reach   delay   offset  jitter

Yes, you are advertising as S1, where it's documented that the refid
will be interpreted as up to 4 printable ascii bytes.  The easy solution
here is to make sure the stratum is at least 2 in this case.

> when they see things like the above.

The above confusion will be gone when the refid smear is accompanied
with a stratum other than S1.

> should know how to handle this and display something useful instead of

See above.  I'm open to ideas on how to display this.

> the code is is not useful at all, IMO.

OK.  I consider not being able to identify when a time source is
providing smeared time to be dangerous.

>> smeared time.  With this proposal it becomes obvious.

Yes.  See above (about S2+).

> Is 4.2.8p3 the old code or the new code?

Old - the new code is the fix for bug 278, which has not yet gone public.

But again, it doesn't matter - no versions of ntpd currently do anything
with either 254. or 255. refids.

> assumes the server is smearing a leap second, and tries to undo the

If we're using an IPv6 address for this association and the packet's
refid matches a.b.c.d or 255.b.c.d then it's a loop.  Otherwise it's not
a loop.

In the case you describe, the cool new client that wants to undo a smear
correction will probably need to consider several things:

- is the refid for this association changing from one 254. refid to
another?  If not, then it's not a smearing refid.

- is this a day where a leap second can be applied?

- do we have a valid leapfile?

Then there are other considerations.

How much do we want to worry about people who update their version of
ntpd everywhere except their time server machines?

I'm fine taking reasonable steps to deal with rare poorly-implemented
configurations when these steps do not compromise our ability to deliver
better time to the vast majority.  Yes, we need to be deliberate in our
choices and be as widely useful as possible.  If there are better ideas
out there let's see them.  I see disseminating the smear in an extension
field as much weaker mechanism when compared to the 254. refid proposal
- it's *much* simpler and it's also more visible.

---

From: Harlan Stenn <[email-masked]>
Date: Mon, 20 Jul 2015 11:22:40 -0700

> the primary purpose of the refid field.

Please provide a better solution then.

We have ongoing and increasing complaints about collisions between IPv6
hashes and IPv4 addresses in the refid.

This is a trivial patch that can be easily backported by vendors who
continue provide really old versions of ntpd to their customers.

How many installations are out there where this sort of timing loop
would actually be an issue?  I think it's a *very* small number.
Remember that if we don't detect this timing loop the stratum values and
root distance will increase, so it will soon become evident that there
is a problem with the local configuration.

---

From: Harlan Stenn <[email-masked]>
Date: Mon, 20 Jul 2015 11:31:45 -0700

> suggestion to transport the smear by means of an extension field.

The older NTP clients won't care - they'll get smeared time and we can
hope/expect that they will ignore the unknown extension field (as
opposed to dropping the packet and then not getting time at all).

How will people running older software know that they are getting
smeared time?

This must be clearly obvious information.

How can this be clearly obvious if the information is in the time packet
but isn't reflected anywhere in the running instance of ntpd?

H
--

---

From: Hal Murray <[email-masked]>
Date: Mon, 20 Jul 2015 11:40:09 -0700

> neither complete, nor mandatory: 

Just to make sure we are on the same wavelength...

Thanks for the list.  I think they are orthogonal to the smear/IPv6 
discussion.

...
>      | GOES | Geosynchronous Orbit Environment Satellite               |
...

Those REFID codes are only used for stratum 0 refclocks and thus show up on 
replies from stratum 1 servers.

> Beside this, please keep in mind that ntpd uses a number of special  refid
> strings internally. A quick grep for sys_refid yields e.g. "STEP",  "DOWN",
> "INIT", and "LOOP". 

Those are only used for stratum 16 servers, aka non-working servers.  (16 
turns into 0 on the wire)

-- 
These are my opinions.  I hate spam.

---

From: Hal Murray <[email-masked]>
Date: Mon, 20 Jul 2015 11:54:35 -0700

[email-masked] said:
> I think Google does it (= smear leap seconds, not change the colors).
> Microsoft Windows smears the time in longer intervals all the year (AFAIK).
> Fortunately Windows does not provide NTP services. Now what if a user wants
> a "always smear time" mode? 

I think there is a terminology problem here.  Or maybe I'm just confused.

In this context, "smear" refers to Google's hack to avoid problems of leap 
seconds.

I expect the NTP term for what Windows does is "slew" rather than "smear".  
(There are no leap-seconds "all the year", so smear doesn't make sense.)

---

From: Hal Murray <[email-masked]>
Date: Mon, 20 Jul 2015 12:22:04 -0700

> identify an NTP server which is serving UT1 instead of UTC?

That makes sense to me.  That changes the REFID hack to mean this-server-lies 
rather than this-server-is-google-leap-smearing.

Of course, the REFID changes we are discussing are only hacks to help humans debug/monitor things.

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 05:10:13 +0000

brian utterback writes:
> second to get it right, as long as we don't drop it. So, assuming we
> are working NTPv5, what would we do? Then once we have an idea of the
> right thing to do in an ideal world, how close can we get in NTPv4?
> And once we know that, what can we do about legacy software? I think
> that this is the proper order. Working the other way means we might
> just make the problem harder down the road.

For NTPv5, I'd like to get NTF's General Timestamp API a bit more
evolved, as I'd like to have the timescale included in the packet
information.  It would be nice if the client specified the timescale to
be used and the server responded using the same (version of the same)
timescale, and it will take more study before I'd want to start nailing
things down.

For NTPv4, it looks like I've got my work cut out for me to convince
folks that the refid changes are good.  I'm pretty comfortable with
them, and I know I could be missing something.

It seems to me that if people are wedded to "all old software must work
the way I want" and "all existing configurations must continue to work
even if one side updates its software" then we're doomed to failure.

As for leap smearing, I'm still trying to understand why people think:

ntp.a.conf: server b

ntp.b.conf: server a

is a useful configuration, as opposed to:

ntp.a.conf: peer b

ntp.b.conf: peer a

-- 
Harlan Stenn <[email-masked]>
http://networktimefoundation.org - be a member!
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 05:26:29 +0000

> http://www.nist.gov/pml/div688/grp40/ut1_ntp_description.cfm

That should be easy.  We could use 253.b.c.d for this case.  It won't
work for the S1 case though.  For that, the server should either claim
S2 status (no great loss), or send .UT1. or similar as its refid.  In
this case, however, there is no automatic way to instruct the S2 server
to use a refid that indicates the S1 it's following is a UT1 source.

This is a timescale issue that is addressed by the GTSAPI and would, I
expect, be conveyed in NTPv5 packets.

For NTPv4, there's sitll the issue of the deeper stratum machines that
sync to a machine that is getting smearing time.

If the S2 machine is offering 254. or 253. refids (saying "I'm smearing
time" or "I'm serving UT1") then the S3 machines that talk to it would
still use a refid based on the IP address of the S2 it's getting time
from.

If we changed that, then if folks were not careful about their server
topology we wouldn't be able to detect timing loops.  But I still think
this is a self-correcting problem because in this case each poll
interval the other side would "ratchet down" the stratum levels.

For peer associations, one won't see 254. refids, as smeared time is
only offered to client associations.
-- 
Harlan Stenn <[email-masked]>
http://networktimefoundation.org - be a member!
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: "Ulrich Windl" <[email-masked]>
Date: Tue, 21 Jul 2015 08:08:41 +0200

Hi!

Well, "smear" is not a technical term with clear semantics, so the obvious interpretation is anything that changes the clock frequency from the correct frequency IMHO.
Is there a true technical specification of "smear" or "smeard lepsecond"?

Regards,
Ulrich

>>> Hal Murray <[email-masked]> schrieb am 20.07.2015 um 20:54 in

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 09:46:31 +0200

Ulrich,

> Is there a true technical specification of "smear" or "smeard lepsecond"?

I have not seen a technical specification, but I think in the context of 
this discussion "slew" means that a client adjusts its system time 
gradually, to apply a leap second or another correction, the slewed 
system time is sent out in reply to client requests, and also other 
applications running on the slewing machine see the slewed system time.

Contrarily, "smearing a leap second" as I've implemented it for ntpd 
means that all the internal timing is unchanged, i.e. ntpd passes the 
leap second warning to the kernel, the kernel steps the time back to 
insert a leap second, etc. Only the time put into the reply packets on 
client requests is modified by a "smear offset" which increases from 0 
to -1 s over the smear interval, so that the sent time matches UTC again 
after the end of the leap second.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 21 Jul 2015 10:02:21 +0200

> hashes and IPv4 addresses in the refid.

You mean users are confused when they see an IPv6 based refid in ntpq
or other utilities thinking it's an IPv4 address? The solution is
simple, stop printing it in the quad dotted notation. If it was
printed as 56:AB:CD:EF for example, it would be clear it's not an IPv4
address. The world is moving to IPv6. IPv4 shouldn't have any special
treatment in NTP.

As for detecting a smearing server, you can pick a fixed refid for
that. If you set it to 127.127.1.255, ntpq would print it as
"LOCAL(255)", as if it was a local driver, which is usually not
serving true time and users are already familiar with it.

If you really need to transfer the smearing offset, instead of abusing
the refid field add an extension field. But if you could modify the
client to work with the extension field, why not rather smear the time
locally and can keep serving UTC over NTP? Local slewing is much
better, it can be more accurate as the client knows what's going on.

> continue provide really old versions of ntpd to their customers.

Unless it's an important security issue, I think only few vendors
would be interested in patching the NTP implementation they use.

> would actually be an issue?  I think it's a *very* small number.

Any installation using peering. I don't think it's a small number.

> is a problem with the local configuration.

Yeah, the loop will stop at some point, but it's an undesired state
degrading the quality of timekeeping, which the refid field is
supposed to prevent.

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 21 Jul 2015 10:13:38 +0200

> is a useful configuration, as opposed to:

One reason to prefer the server/client mode over symmetric modes
to peer between two NTP hosts is that it prevents the DoS attack on
(unauthenticated) symmetric associations. Another reason could be that
some configuration tools/mechanisms only support the server directive.

---

From: <[email-masked]>
Date: Tue, 21 Jul 2015 08:54:54 +0000

Hi,

I agree. A smeared second is not telling you what the time is. NTP already handles leap seconds, so there is no need for this awful behaviour. An end client may choose to step, repeat, smear the second or even use 23:59:60 properly. That though is not NTP's concern.

Regards,

Tony Flavin

-----Original Message-----
From: ntpwg [mailto:ntpwg-bounces+anthony.flavin=[email-masked]] On Behalf Of Ulrich Windl
Sent: 20 July 2015 12:57
To: [email-masked]; [email-masked]; [email-masked]
Subject: [ntpwg] Antw: Re: Proposed REFID changes

Hi!

I cannot resist the imagination of Dave Mills' face chaging color three times per seconds when reading about smeared leap seconds ;-) I think Google does it (= smear leap seconds, not change the colors). Microsoft Windows smears the time in longer intervals all the year (AFAIK). Fortunately Windows does not provide NTP services. Now what if a user wants a "always smear time" mode?
That's not NTP any more IMHO.

Regards,
Ulrich

<[email-masked]>:
>>> The attached change goes to an additional way to flag a refid at S2+ 
>>> as being an IPv6 refid.
>>>
>>> unsigned fractional seconds.  This value is the amount of smear that 
>>> has been applied to the timestamps in the packet.  But I see I 
>>> haven't finished that writeup yet.
>> 
> I don't buy the argument that this will cause timing loops.  If there 
> is no timing loop when there isn't a leap second, why would there 
> suddenly be a timing loop during the leap second if we used 254.a.b.c 
> to identify that we're offering smeared time
> 
>>> Newer versions of ntpd (and people) can also use this to detect when 
>>> a time server is sending them smeared time.
>>>
>>> I believe these changes are safe because initial octets of 254 and 
>>> 255 are not permitted as source addresses in an IPv4 address.
>> 
> regular time or smeared time.  The 254.a.b.c is currently used more 
> for humans than ntp.  This can change in the future, so if one gets 
> into the situation where there are 2 smearing servers and 2 normal 
> servers, we'd be able to "reverse-engineer" the smear amount and come 
> up with the correct time, and have a better chance of finding a majority clique.
>

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 10:58:34 +0200

Harlan,

Harlan Stenn wrote:
>>>>> has

Maybe. But IMO fiddling with the refid is *not* a proper way.

> extension field know that time from that server is being smeared?

You know we (Meinberg) are selling NTP time server appliances, the 
LANTIME devices. There have always been requests from customers to let 
the NTP servers transmit raw GPS time, TAI, and even local time.

I *know* that especially the latter is strongly non-recommended, but 
there are special cases where this makes sense. For example, one of our 
customers uses an LANTIME serving local time on a cruise ships, and the 
only clients are displays showing that time as wall clock time. So if 
cruise ship crosses a time zone border the admin only has to change the 
time zone offset on the LANTIME, and all the displays switch to the new 
local time accordingly.

I know this is not the original idea how NTP should work, but it saves 
the user from huge efforts to reconfigure each individual display, and 
of course the user needs to be aware that this NTP server can't be used 
to synchronize computers to UTC time. For this standard purpose they 
have another NTP server in standard configuration, serving UTC time.

It's the same with NTP servers sending TAI or GPS time instead of UTC. 
This eventually makes sense for special applications in closed networks, 
but of course such a machine must not be used as public time server.

> Also, I think we *must* add the smear to the root dispersion.

I disagree here. The smearing approach has been introduced as a 
*workaround* to avoid problems on systems where the *kernel* inserts a 
leap second by just stepping the system time back by 1 second. The 
stepped back time causes trouble with many applications because there 
are duplicate time stamps around the leap second, and thus ordering 
events based on time stamps may not work as expected.

The "leap smearing" approach is a perfect solutions for local company 
networks where it is just most important that the time seen by 
*applications* never steps back. Letting your own company NTP server 
care about this by "smearing" a leap second for the whole network 
requires *much* less effort than taking care that each client node has 
been configured to run ntpd with -x (slew always), making sure a version 
of ntpd is running which *really* slews leap seconds instead of stepping 
the time in spite of the -x flag, etc. See the discussion in
http://bugs.ntp.org/show_bug.cgi?id=2745

So smearing explicitly is to fool your *own* NTP clients, so there's no 
reason to let the time server tell the clients that he's *bad*.

> applied over the next period.  That will limit the error to at most a

*If* you are using leap smear, do you really think it matters much if 
the maximum error is 1 s or 0.5 s??

If the smearing is finished at the end of a leap second then the time is 
immediately correct again at the end of the leap second. The Japanese 
stock exchange folks *have* smeared the time before the leap second. 
Please ask them if they would have preferred that the time on their 
machines would have been off by only 0.5 s at max, but still for several 
hours after the leap second when they had already started their usual 
business.

> here is to make sure the stratum is at least 2 in this case.

Yes, of course. And when I do this on our LANTIMEs then a normal refid 
like "PPS" or "GPS" would be shown as garbage, or a reverse DNS lookup 
would be tried all the other time when the S1 server is *not* smearing?

Please don't tell me now that we should also start to fiddle with the 
stratum, change if from S1 to S2, as long as smearing is in effect. :-(

> with a stratum other than S1.

And if you have a look at the overall picture you'll see that all this 
doesn't really help if servers configured as clients of smearing servers 
start in turn to supply time to *their* clients.

In this case the fudged refid doesn't arrive at the next layer of 
clients anyway, so they don't know anyway that they receive "smeared" time.

The only real solution is that admins know what they do when they set up 
the time service for their corporate network.

If the set up DNS or DHCP incorrectly then this will also not work as 
expected, and there will always be folks who don't care about a proper 
setup.

In any case the fiddling with the refid causes more confusion than it 
helps since not even the current version of ntpq knows how to deal with 
this correctly. :-(

> providing smeared time to be dangerous.

Smearing is not intended for public servers, but it is very helpful for 
folks providing their own networks with time from their own servers. 
This *can* be local stratum 2 servers which get the time from the pool, 
but many companies are buying their own NTP servers, e.g controlled by 
GPS, in which case they are stratum 1.

So again, this fiddling with the refid while smearing causes only 
confusion, as we have een above.

> Yes.  See above (about S2+).

This is clearly not a solution but causes confusion. :-(

> with either 254. or 255. refids.

Except that garbage is displayed. :-(

> correction will probably need to consider several things:

In my opinion no client should try to undo the smearing.

In the normal case smearing is explicitly configured on the server just 
to let clients use the smeared time.

If an admin has configured some smearing servers and some non-smearing 
servers then ntpd should simply accept the majority when smearing starts.

1.) If the non-smearing servers are the majority then ntpd will accept 
the leap second warning from those servers, mark the smearing servers as 
falsetickers, and insert the leap second as usual, so the leap second is 
applied in the usual way.

2.) If the smearing servers are the majority then ntpd should *not* 
accept a leap second warning from the non-smearing servers since 
smearing servers don't provide a leap second warning, and thus there's 
*no* majority of time sources announcing a leap second. Instead, the 
non-smearing servers should be marked as false tickers and the client 
should just follow the smearing servers and slew their system time 
slowly to account for the leap second.

In either case the client will have the correct UTC time immediately 
after the leap second.

Now if you start fiddling with the root dispersion on smearing servers, 
what is going to happen? The clients may prefer the non-smearing servers 
over the smearing servers, but are you sure they will also accept the 
leap second warning flag? Or will this result in some unspecified 
behaviour of the client?

Right now I'm talking about how it *should* be, but how it *would* be 
with existing clients which can be 4.2.8, 4.2.6, 4.2.4, or even older.

I know that you could say now that customers should upgrade their NTP 
software, but believe me we have many customers out there which don't 
want or just can't do this due to some policies.

> - is this a day where a leap second can be applied?

Is there an IPv6 hash starting with 254 at this day, or only on other days?

> - do we have a valid leapfile?

For the smearing server it's fine, for a "smeared" client it's not, 
regardless of the refid.

> Then there are other considerations.

Yes. See above.

> - it's *much* simpler and it's also more visible.

*If* we wanted to introduce a smeared refid then we should have done 
this consistently in a given packet version, after thoroughly having 
thought about the consequences, so that ntpq showed something useful 
instead of garbage, at least clients running the same version would 
ignore the leap second file, etc.

Anyway, this would at most help at the first level of clients of a 
smearing server, but not at all for the client's clients, unless you 
pass the "smear" refid down the chain.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 11:10:46 +0200

[email-masked] wrote:
> Hi,
>
> I agree. A smeared second is not telling you what the time is. NTP
> already handles leap seconds, so there is no need for this awful
> behaviour.

Smearing leap seconds is an excellent way to avoid problems with leap 
seconds in an easy and straight forward way.

This doesn't happen automatically. You explicitly have to build ntpd 
with smearing support compiled in, and even then you explicitly have to 
configure it.

We have quite a number of customers who appreciate very much that ntpd 
now provides this feature since it solves a lot of potential problems 
they had before when a leap second was to be inserted.

 > An end client may choose to step, repeat, smear the second
 > or even use 23:59:60 properly. That though is not NTP's concern.

And if you are responsible you have to take care that every single 
client does excactly what you want, depending on the software version, 
the client configuration, etc.

With smearing you just have to take care that your server(s) are 
configured appropriately, and all clients just follow the smeared time 
regardless of the software version, just because they don't even notice 
there's a leap second.

Believe me, there are many admins out there who prefer (and can accept) 
a temporary 1 s time offset over the risk of running a version of ntpd 
or the OS kernel on any of their machines which doesn't handle a leap 
second correctly.

If you think you don't need it, just don't use it.

Regards,

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 11:14:13 +0200

> replies from stratum 1 servers.

Yes, thanks!

> turns into 0 on the wire)

Indeed. But anyway it may be useful to mention these codes in the RFC.

Regards,

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 11:17:01 +0200

> How will people running older software know that they are getting

Do they have to? Or is it sufficient for the IT admin to know that he 
has set up a smearing server for the company?

As I've said earlier, several times: this is *not* feature to be used 
with public/pool servers.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 02:41:09 -0700

> has set up a smearing server for the company?

I'd rather not hide it.  And at this point the only folks who will see
it are the folks talking directly to the server that is doing the
smearing, because any "deeper" NTP instances will have a refid that
points to the server being "fed" smeared time.

> with public/pool servers.

Sure.  And it will happen anyway.

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 02:43:38 -0700

I agree with Martin.  This is a mechanism/policy thing.  There needs to
be a robust mechanism to allow it (and for me that includes the REFID
thing, to make it easy to see when a client has sync'd to a machine
providing smeared time), and then it's a local policy choice about
whether or not somebody wants to use those servers or not.

H

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 02:54:42 -0700

> address. The world is moving to IPv6. IPv4 shouldn't have any special

There is no way to tell.  The refid shows up as a 32-bit number, with no
information to let the receiving side know what sort of IP address the
sys_peer was.

> serving true time and users are already familiar with it.

Then we'd also have to start encoding the "type" of smear being applied,
unless we started sending the smear value another way.

> better, it can be more accurate as the client knows what's going on.

That's great if it can be done.  The problem we need to solve is
providing smeared time to old clients.  It's easy to provide the smeared
time.  It must, I believe, also be easy to figure out if we're following
a server that is getting smeared time.  This may not be something we can
determine locally.  We might need to use ntptrace or something to find out.

> would be interested in patching the NTP implementation they use.

That's their choice too.

> Any installation using peering. I don't think it's a small number.

How do we find out?

> degrading the quality of timekeeping, which the refid field is

If there are better sources of time then as the two looped machines
spiral down then these better sources of time will be chosen.  Loop broken.

I think we really need to see if this is a problem.  I really don't
think it is.

But we might want to support multiple mechanisms for identifying if and
how smearing is done, and then leave it as a local policy choice for
what folks want to do.

The best answer, I think, is NTPv5 using the GTSAPI stuff, as that
includes a timescale field.  And getting there from here will take a bit
of work.  Even then, will folks upgrade?

I'd also like to see leap seconds get added/deleted every other month or
so.  Doing this would certainly help weed out the bugs in kernels and
applications.

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 12:07:09 +0200

Harlan,

> points to the server being "fed" smeared time.

Right. So e.g. users of S2 servers in the pool may not see it anyway.

> Sure.  And it will happen anyway.

Bad things happen all the time.

Just recently there have still been servers in the pool which were 
announcing a leap second. As I've explained in an earlier email there 
should not even be a problem if individual servers are mis-configured if 
most of the pool servers get it right.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 21 Jul 2015 12:21:48 +0200

> information to let the receiving side know what sort of IP address the

Right. The client doesn't know if it's an IPv4 address, so don't print
it as an IPv4 address. It doesn't matter if it actually is.

The client doesn't need to know what address it was, it just needs to
compare it to its own reference IDs, which have to be generated in the
exactly same way. That's the purpose of the field.

> unless we started sending the smear value another way.

LOCAL(255) just tells you the server is not serving true time. For an
admin that's the most important fact, either it's expected in that
network or something is wrong and needs to be fixed.

> spiral down then these better sources of time will be chosen.  Loop broken.

If a loop formed between two peers it likely means they both lost
their primary sources of time. As they try to synchronize to each
other before the stratum limit is reached, the frequency of their
clocks is thrown off and they drift away faster from true time than
they would be if the loop was prevented.

---

From: "Ulrich Windl" <[email-masked]>
Date: Tue, 21 Jul 2015 12:24:08 +0200

>>> Martin Burnicki <[email-masked]> schrieb am 21.07.2015 um 10:58 in
Nachricht <[email-masked]>:
[...]
> It's the same with NTP servers sending TAI or GPS time instead of UTC. 

For the TAI, it's rather simple: When using autokey ("why only then?" is a different question) the clients get the "tai=36" information, so you can derive TAI from UTC. (Should apply to GPS also)

[...]

I think the big mistake with NTPv4 was to hack the NTPv3 protocol instead of making it IPv6 clean right at the start. All the hacking with extension fields just created a big mess. MHO...

Regards,
Ulrich

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 12:34:32 +0200

> information to let the receiving side know what sort of IP address the

That's exactly what Miroslav says: stop letting it *look* like an IPv4 
address.

IIRC then formerly even a DNS reverse lookup was tried on the IPv4-like 
refid.  We have had cases where we saw that a DNS reverse lookup was 
even tried for a refid like "PPS". Fortunately this doesn's seem to 
happen anymore, so I'd support Miroslav's suggestion to print the 
numeric refid in a different way.

>> serving true time and users are already familiar with it.

How the refid would be interpreted would still depend on whether the 
server is S1 or S2.

On an S1 it would show "LOCAL(255)", but on an S2 it would show 
"127.127.0.1", or "7F:7F:0:FF" with the proposed new notation.

> determine locally.  We might need to use ntptrace or something to find out.

IMO care should be taken to handle this properly in a new protocol 
version, but it's no good idea to fiddle with existing implementations 
in a way which just causes more confusion.

>> Any installation using peering. I don't think it's a small number.

Do we have to find it out? If any usage of a legacy approach (peering in 
this case) is broken by a patch then we just shouldn't accept that patch.

> so.  Doing this would certainly help weed out the bugs in kernels and

Yes. You should propose this to the IERS folks. ;-))

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 12:47:05 +0200

> For the TAI, it's rather simple: When using autokey ("why only then?" is a different question) the clients get the "tai=36" information, so you can derive TAI from UTC. (Should apply to GPS also)

No. This was intended to work, but didn't work all the time properly 
until recently.

If the server had a leap second file with a leap second entry in the 
future (e.g June 2015) then in May 2015 not the current TAI offset (35) 
but the *future* offset (36) was already sent to the clients, and those 
data sets weren't even accepted by the clients. For quite some time this 
worked only if the last recent entry in the leap second file was already 
in the past. This has only been fixed in 4.2.8p3.

See:
"ntpd doesn't always transfer the correct TAI offset via autokey"
http://bugs.ntp.org/show_bug.cgi?id=2830

> I think the big mistake with NTPv4 was to hack the NTPv3 protocol instead of making it IPv6 clean right at the start.  All the hacking with extension fields just created a big mess. MHO...

I don't think the extension fields themselves are the problem. The specs 
should just have been more clear right from the beginning.

In any case we should now take care not to break compatibility with 
older implementations.

PTP/IEEE1588 is a negative example where PTPv2 followed quickly after 
PTPv1 and is in no way compatible.

In this context NTP is much better since it has kept compatibility over 
*many* years.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 21 Jul 2015 12:50:49 +0200

On Tue, Jul 21, 2015 at 12:34:32PM +0200, Martin Burnicki wrote:
> How the refid would be interpreted would still depend on whether the server
> is S1 or S2.
> 

ntpq seems to interpret the refid as a refclock even when the stratum
is larger than 1. I guess it's like that specifically for the LOCAL
driver.

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+192.168.100.1   LOCAL(255)       2 u  370  512  377    0.100  -14.927   4.069

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 12:57:36 +0200

>       remote           refid      st t when poll reach   delay   offset  jitter

OK, didn't know that.

Thanks,

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Rob Seaman <[email-masked]>
Date: Tue, 21 Jul 2015 06:38:46 -0700

On Jul 21, 2015, at 12:46 AM, Martin Burnicki <[email-masked]> wrote:

> I have not seen a technical specification, but I think in the context of this discussion "slew" means that a client adjusts its system time gradually, to apply a leap second or another correction, the slewed system time is sent out in reply to client requests, and also other applications running on the slewing machine see the slewed system time.
> 
> Contrarily, "smearing a leap second" as I've implemented it for ntpd means that all the internal timing is unchanged, i.e. ntpd passes the leap second warning to the kernel, the kernel steps the time back to insert a leap second, etc. Only the time put into the reply packets on client requests is modified by a "smear offset" which increases from 0 to -1 s over the smear interval, so that the sent time matches UTC again after the end of the leap second.
> 
> Martin

Presumably most here are familiar with this excellent analysis:

	http://developerblog.redhat.com/2015/06/01/five-different-ways-handle-leap-seconds-ntp/

I’ll join Ulrich and Martin in wondering if there are other technical specifications we should consult?

The first coherent description by Markus Kuhn used the term “smoothed leap seconds”, and I think smoothing is a better description of the operation:

	http://www.cl.cam.ac.uk/~mgk25/time/utc-sls/

Perhaps Kuhn’s terminology (and formalism) would make a better foundation for NTP than Google’s?

	http://googleblog.blogspot.com/2011/09/time-technology-and-leaping-seconds.html

Underneath both of these, with each leap second as announced by IERS Bulletin C:

	http://datacenter.iers.org/eop/-/somos/5Rgv/getTX/16/bulletinc-049.txt

There is a corresponding swing in the opposite direction of DUT1, e.g.,

	DUT1 = UT1-UTC went from -0.7s to +0.3s,
	    as UTC-TAI went from  -35s to -36s:

from:	http://datacenter.iers.org/eop/-/somos/5Rgv/getTX/17/bulletind-123.txt
to:	http://datacenter.iers.org/eop/-/somos/5Rgv/getTX/17/bulletind-124.txt

It is more correct to view UTC as a standard in units of tenth-seconds. This is what it provides formally as an approximation to UT1.

So another way to look at smoothing / smearing is as spreading that 10 step adjustment (in tenth-seconds) out over some period of time. At the nominal NTP slew rate of 0.5 ms per second, each tenth-second step would take 200 seconds to accomplish. How NTP (or other software) explicitly implements smoothing of leap seconds (e.g., linear versus cosine, etc.) is a matter for discussion before the next leap second. One hopes Martin’s prototype generated some good field data to analyse from the latest leap second.

Perhaps the word “smear” should be reserved for Google’s implementation? A synonym for smooth is “continuous”, and this is the word the ITU has chosen to emphasize as a key feature needed for UTC. Implementing smoothed / smeared leap seconds is one way to achieve this goal.

Rob Seaman
Network Time Foundation
[email-masked]

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 21 Jul 2015 17:20:51 +0200

Hi Rob,

> 	http://developerblog.redhat.com/2015/06/01/five-different-ways-handle-leap-seconds-ntp/

Haven't seen this page from Miroslav before, but it looks good and 
interesting at the first glance. Will have a closer look later.

> 	http://www.cl.cam.ac.uk/~mgk25/time/utc-sls/

I haven't looked at Markus' page proposal right now, but if I remember 
correctly then this is done at the client side, i.e. when the client 
detects a leap second warning from the server it starts to adjust its 
own system time smoothly over a determined period of time. This means 
the slewed/smoothed time is also visible to applications running on the 
client.

Unfortunately, as far as I know, there are special runtime environments 
which use UTC-SLS:

NASA, What is Common Data Format (CDF)?
http://cdf.gsfc.nasa.gov/

NASA, Requirements for handling leap seconds in CDF
http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html

Andrew Main (Zefram)'s Time::UTC_SLS package for perl
http://search.cpan.org/~zefram/Time-UTC_SLS-0.003/lib/Time/UTC_SLS.pm

Oracle's Java: java.time.Clock class
https://docs.oracle.com/javase/8/docs/api/java/time/Clock.html

However, there is no implementation for the kernel, so applications 
getting the time directly from the kernel still suffer from the time 
being stepped back by 1 s at the beginning of an inserted leap second.

> 	http://googleblog.blogspot.com/2011/09/time-technology-and-leaping-seconds.html

Google's approach has the advantage, IMO, that the slewing doesn't have 
to be implemented on each client. The current implementation in ntpd 
4.2.8p3 works like the Google folks have originally described it in the 
blog quoted above:

The smearing starts a determined time before the leap second, has a 
cosine shape, and is finished at the end of the leap second.

The difference to UTC-SLS is that clients don't even notice that a leap 
second is inserted, and all clients follow the smeared time transmitted 
by the server in pretty much the same way.

In the current implementation in ntpd 4.2.8p3 *only* the time sent in 
reply to client requests is smeared with a well defined offset, while 
the internal system time, time comparison with NTP peers, and with 
refclocks is *not* affected, i.e. applications running on the smearing 
server would observe a time step back at the beginning of a leap second 
instead of a smeared time.

IMO this is most useful for devices acting as NTP servers for a company 
network, not for the servers and workstations in that networks.

So this is a fundamental difference to UTC-SLS.

I know Google has implemented the smearing differently this time, using 
a linear instead of cosine slope, and smearing across a symmetric 
interval before and after the leap second. However, I don't see the real 
advantage of the new approach.

I'm currently running some tests to see how standard NTP clients behave 
when they get smeard time in a cosine vs. linear shape, over diffferent 
smear intervals and with different polling polling intervals.

> to:	http://datacenter.iers.org/eop/-/somos/5Rgv/getTX/17/bulletind-124.txt

Right that's basically the same as if you look at the time difference 
shortly before or after the beginning of a leap second, where the time 
difference changes from x to (1 s - x).

> It is more correct to view UTC as a standard in units of tenth-seconds. This is what it provides formally as an approximation to UT1.

Hm, has a resolution of 100 ms specifically been defined somewhere? For 
me this sounds like even more problems if you try to do an adjustment in 
several steps of 100 ms instead of a single one.

> So another way to look at smoothing / smearing is as spreading that
> 10 step adjustment (in tenth-seconds) out over some period of time.
> At the nominal NTP slew rate of 0.5 ms per second, each tenth-second
> step would take 200 seconds to accomplish. How NTP (or other
> software) explicitly implements smoothing of leap seconds (e.g.,
> linear versus cosine, etc.) is a matter for discussion before the
> next leap second. One hopes Martin’s prototype generated some good
> field data to analyse from the latest leap second.

If I imagine the Linux kernel which steps the time back by 1 s now, and 
the kernel would do 10 steps back by 100 ms each during an inserted leap 
second I don't know if that would result in less problems than we have 
today.

If the steps are inserted over a longer interval then we've basically 
once more some new kind of slewing, and the UTC time until the beginning 
and after the end of the leap second will not be accurate.

> Perhaps the word “smear” should be reserved for Google’s implementation?

That's what ntpd 4.2.8p3 currently does, if it's used. ;-)

> A synonym for smooth is “continuous”, and this is the word the ITU
> has chosen to emphasize as a key feature needed for UTC. Implementing
> smoothed / smeared leap seconds is one way to achieve this goal.

IMO the best solution would be to fix the kernels so that they don't 
step the time back. In this case we'd need neither slewing nor smearing 
to be implemented in ntpd and other time synchronization software.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Warner Losh <[email-masked]>
Date: Tue, 21 Jul 2015 10:15:06 -0600

> On Jul 21, 2015, at 9:20 AM, Martin Burnicki <[email-masked]> wrote:
> 
> Hi Rob,
> 
> Rob Seaman wrote:
> 
> Hm, has a resolution of 100 ms specifically been defined somewhere? For me this sounds like even more problems if you try to do an adjustment in several steps of 100 ms instead of a single one.

I don’t buy that “it is more correct to view UTC as a standard in units of tenth-seconds.” There’s no real justification for that statement in the standards that define UTC, and at best weak justification in IERS’ publication of the current UT1-UTC offset accurate to .1s on one of its many publications (bulletin D). That value was used for broadcast time, but last I interacted with broadcast time (e.g. WWV) it was being phased out because there were so few users of it (everybody that care about UT1 cares at a might higher level, so they use different, more accurate sources instead of the WWV). IERS publishes UT1-UTC to a much higher accuracy elsewhere (Bulletin A and B). Values to the microsecond level, with error bars in the 10us level.

The only values that are talked about in the UTC standard are that UTC will be an approximation of UT1, and the approximation will be within 0.9s of UT1.

Warner

---

From: Warner Losh <[email-masked]>
Date: Tue, 21 Jul 2015 10:19:02 -0600

> IMO the best solution would be to fix the kernels so that they don't step the time back. In this case we'd need neither slewing nor smearing to be implemented in ntpd and other time synchronization software.

I agree. Either step time back by a second to overcome time_t’s inability to encode leap seconds, or assume a 1second error will be introduced at the known leap second time and average that error out by one of the many soothing algorithms. 100ms steps are insane, and give races around time steps 10x the chance to be lost badly.

The exact decision, though, is a local policy one, not something that can be set for everybody. There are systems out there that really want to have the system clock have as small an error as possible from UTC and know how to cope with the weirdness around the leap second. There are other systems that have no clue that a leap second exists and have relaxed enough timing requirements that the smoothing of this induced error works better.

Warner

---

From: Rob Seaman <[email-masked]>
Date: Tue, 21 Jul 2015 10:01:15 -0700

> The only values that are talked about in the UTC standard are that UTC will be an approximation of UT1, and the approximation will be within 0.9s of UT1.

My email was to suggest the word smoothing rather than smearing, and to start to gather a list of pertinent technical documents as suggested by Martin and Ulrich.  One of those documents should obviously be:

	https://www.itu.int/rec/R-REC-TF.460-6-200202-I

...in which the term DUT1 is mentioned 27 times.  This is the normative document and the IERS Bulletins are responsive to explicit requests with wording like:

>>>> C Coordinated universal time (UTC)
>>>> 
>>>> UTC is the time-scale maintained by the BIPM, with assistance from the IERS, which forms the basis of a coordinated dissemination of standard frequencies and time signals. It corresponds exactly in rate with TAI but differs from it by an integer number of seconds.
>>>> 
>>>> The UTC scale is adjusted by the insertion or deletion of seconds (positive or negative leap- seconds) to ensure approximate agreement with UT1.
>>>> 
>>>> D DUT1
>>>> 
>>>> The value of the predicted difference UT1 – UTC, as disseminated with the time signals is denoted DUT1; thus DUT1 ≈ UT1 – UTC. DUT1 may be regarded as a correction to be added to UTC to obtain a better approximation to UT1.
>>>> 
>>>> The values of DUT1 are given by the IERS in multiples of 0.1 s.
>>>> 
>>>> The following operational rules apply:
>>>> 
>>>> Rec. ITU-R TF.460-6 3
>>>> 
>>>> 1 Tolerances
>>>> 
>>>> 	• 1.1  The magnitude of DUT1 should not exceed 0.8 s.
>>>> 
>>>> 	• 1.2  The departure of UTC from UT1 should not exceed ±0.9 s (see Note 1).
>>>> 
>>>> 	• 1.3  The deviation of (UTC plus DUT1) should not exceed ±0.1 s.
>>>> 
>>>> NOTE 1 – The difference between the maximum value of DUT1 and the maximum departure of UTC from UT1 represents the allowable deviation of (UTC + DUT1) from UT1 and is a safeguard for the IERS against unpredictable changes in the rate of rotation of the Earth.

and:

>>>> 2 Leap-seconds
>>>> . . .
>>>> 2.3 The IERS should decide upon and announce the introduction of a leap-second, such an announcement to be made at least eight weeks in advance.
>>>> 
>>>> 3 Value of DUT1
>>>> 
>>>> 3.1 The IERS is requested to decide upon the value of DUT1 and its date of introduction and to circulate this information one month in advance. In exceptional cases of sudden change in the rate of rotation of the Earth, the IERS may issue a correction not later than two weeks in advance of the date of its introduction.
>>>> 
>>>> 3.2 Administrations and organizations should use the IERS value of DUT1 for standard-frequency and time-signal emissions, and are requested to circulate the information as widely as possible in periodicals, bulletins, etc.

Bulletins C and D are coordinated such that an adjustment to DUT1 is made at the same time as a leap second and in the opposite direction.

DUT1 / Bulletin D cannot be phased out and remain in conformance with ITU TF.460-6. If TF.460 is changed at WRC15 the new standard (whichever variation) has been proposed to take effect 5-10 years after passage, so there will likely be at least two or three future leap seconds under all scenarios. Smoothing of the leap second will be put to the test.

Rob

---

From: Warner Losh <[email-masked]>
Date: Tue, 21 Jul 2015 12:33:33 -0600

> On Jul 21, 2015, at 11:01 AM, Rob Seaman <[email-masked]> wrote:
> 
> My email was to suggest the word smoothing rather than smearing, and to start to gather a list of pertinent technical documents as suggested by Martin and Ulrich.

Right. My main issue was ‘in units of tenth-seconds’. Because one product of the IERS has .1s resolution doesn’t make the UTC standard a standard of tenth-seconds.

> One of those documents should obviously be:
> 

Yes. I didn’t know of a authoritative reference for this since the last time I found the document it was on a warez site and was told (at the time) it cost big $$$ to get legitimately. Glad to see it freely available.

> DUT1 / Bulletin D cannot be phased out and remain in conformance with ITU TF.460-6.

The values I was talking about were the broadcast values in WWV. While that would take it out of conformance with TF.460, I was told at the time it was no longer important. I don’t know if those plans were ever implemented. It appears that they are still broadcast, so perhaps those conversations went no where, despite coming from folks high up in the NIST timekeeping community.

> If TF.460 is changed at WRC15 the new standard (whichever variation) has been proposed to take effect 5-10 years after passage, so there will likely be at least two or three future leap seconds under all scenarios. Smoothing of the leap second will be put to the test.

True. I’m not sure WRC15 will actually do anything (again), but that’s likely beyond the scope of this discussion.

Warner

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 21 Jul 2015 14:22:09 -0700

On 7/21/15 3:24 AM, Ulrich Windl wrote:
> Nachricht <[email-masked]>: [...]
> For the TAI, it's rather simple: When using autokey ("why only then?"
> is a different question) the clients get the "tai=36" information, so
> you can derive TAI from UTC. (Should apply to GPS also)

Nobody should be using autokey.  Or from the other direction, if you are
using autokey you should stop using it.  If you think you need to use it
please email me and tell me your story.

I think we're getting a bit offtopic for the WG list.

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 22 Jul 2015 08:20:07 +0200

>>> Martin Burnicki <[email-masked]> schrieb am 21.07.2015 um 17:20 in
Nachricht <[email-masked]>:
[...]
> second I don't know if that would result in less problems than we have 

What are these kind of problems? Badly designed application?
There is POSIX clock_gettime(CLOCK_MONOTONIC, ...) (among others, since 1993) that provides a linear timescale for measurements that won't be affected by leapseconds or other time zones (In Linux it will be affected by reboots however as it seems it's the time since reboot).

[...]
> step the time back. In this case we'd need neither slewing nor smearing 

But kernels don't step the time back; they insert an extra 23:59:60 second, right?

[...]

So what are the real problems?

Regards,
Ulrich

---

From: Hal Murray <[email-masked]>
Date: Tue, 21 Jul 2015 23:43:55 -0700

> But kernels don't step the time back; they insert an extra 23:59:60 second,

Linux kernels set the time back a second to repeat 23:59:59

> So what are the real problems?

One of the problems is that POSIX pretends that leap seconds don't exist.

There is no way to talk about the leap seconds that get inserted.  The kernel 
is keeping POSIX time.

---

From: <[email-masked]>
Date: Wed, 22 Jul 2015 07:25:51 +0000

There is really some bad science going on here.

So a client that may well have backed off to 1024 sec polling (V3 or greater for V4) will see a server announce a leap second. But then over a not well specified interval (let's say an hour) effectively become a false ticker. The client will only poll three times in this period itself, which is not enough for it to start taking action anyway.

It would make more sense to create a client profile that can act on the existing leap second flag and then "smear" time to its host system, rather than break the server architecture and propagate the wrong time.

Tony Flavin

-----Original Message-----
From: ntpwg [mailto:ntpwg-bounces+anthony.flavin=[email-masked]] On Behalf Of Rob Seaman
Sent: 21 July 2015 14:39
To: Martin Burnicki; Ulrich Windl
Cc: [email-masked]; Hal Murray
Subject: Re: [ntpwg] Proposed REFID changes

I'll join Ulrich and Martin in wondering if there are other technical specifications we should consult?

The first coherent description by Markus Kuhn used the term "smoothed leap seconds", and I think smoothing is a better description of the operation:

Perhaps Kuhn's terminology (and formalism) would make a better foundation for NTP than Google's?

So another way to look at smoothing / smearing is as spreading that 10 step adjustment (in tenth-seconds) out over some period of time. At the nominal NTP slew rate of 0.5 ms per second, each tenth-second step would take 200 seconds to accomplish. How NTP (or other software) explicitly implements smoothing of leap seconds (e.g., linear versus cosine, etc.) is a matter for discussion before the next leap second. One hopes Martin's prototype generated some good field data to analyse from the latest leap second.

Perhaps the word "smear" should be reserved for Google's implementation? A synonym for smooth is "continuous", and this is the word the ITU has chosen to emphasize as a key feature needed for UTC. Implementing smoothed / smeared leap seconds is one way to achieve this goal.

Rob Seaman
Network Time Foundation
[email-masked]

---

From: Brian Utterback <[email-masked]>
Date: Wed, 22 Jul 2015 08:45:09 -0400

> Linux kernels set the time back a second to repeat 23:59:59

So does Solaris.

> There is no way to talk about the leap seconds that get inserted.  The kernel

This is why I think we should start by looking from the ground up at an 
ideal world, then once that is established, work towards it. There are a 
hundred intermediate goals that we each might take as our own goal and 
we will end up working at cross-purposes.

All working systems eventually show their own agendas.

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 22 Jul 2015 15:12:28 +0200

>>> Hal Murray <[email-masked]> schrieb am 22.07.2015 um 08:43 in

> There is no way to talk about the leap seconds that get inserted.  The 

Thanks, I see: The POSIX time structs don't have a leap indicator, and adding it might break some badly designed applications (relying on the size of the struct).

---

From: Warner Losh <[email-masked]>
Date: Wed, 22 Jul 2015 09:05:49 -0600

> On Jul 22, 2015, at 7:12 AM, Ulrich Windl <[email-masked]> wrote:
> 

time_t is typically a simple integer of some kind (though some oddball floats exist). There’s no room to add a leap indicator. And the formula from the standard for the value of the time_t precludes them entirely. Encoding the leap second using the standard formula gives a time_t value that’s the same as the following second. That’s why time steps backwards on kernels that do leap seconds: they have to repeat some value. Some repeat the last second of the day (since they want to properly have the leap second be part of same day), while others repeat the first second of the next day (since that’s what naive math from the POSIX standard might[*] imply).

time_t has even changed size on a number of recent open source systems without breaking applications at the source level. At the binary level, every application on the planet encodes the side of an integer in them. That doesn’t make them badly designed, that’s just how C works. I’d go so far as to say that relying on the size of the struct is also not badly designed, since that’s supposed to be a run-time invariant.

It goes well beyond the size of the integer. The semantics of the value of time_t preclude leap seconds. The semantics of C harden sizes of things at compile time.

Warner

[*] I said might here, since blindly applying the formula does give this result. Yet other parts of POSIX are quite emphatic that leap seconds are outside the scope of things that are standardized. The whole issue of leap seconds is quite contentious within POSIX with one side eventually kicking out the other and winding up with the infamous ‘double leap second’ value of 61 sneaking into old versions of the standard.

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 22 Jul 2015 17:19:49 +0200

Warner Losh wrote:
>> IMO the best solution would be to fix the kernels so that they
>> don't step the time back. In this case we'd need neither slewing
>> nor smearing to be implemented in ntpd and other time
>> synchronization software.
>
> I agree. Either step time back by a second to overcome time_t’s
> inability to encode leap seconds, or assume a 1second error will be
> introduced at the known leap second time and average that error out
> by one of the many soothing algorithms. 100ms steps are insane, and
> give races around time steps 10x the chance to be lost badly.

That's exactly the problem I see.

> The exact decision, though, is a local policy one, not something that
> can be set for everybody. There are systems out there that really
> want to have the system clock have as small an error as possible from
> UTC and know how to cope with the weirdness around the leap second.
> There are other systems that have no clue that a leap second exists
> and have relaxed enough timing requirements that the smoothing of
> this induced error works better.

Yes, and both are now using the same system clock, CLOCK_REALTIME.

So if an alternative API were to be introduced then

- either CLOCK_REALTIME had to be modified to return slewed time without 
a step. In this case applications which get confused by time steps back 
would work properly, but applications which expect exact UTC time might 
fail,

- or a new API could be introduced which returns slewed system time 
without a step back, but in this case applications which get confused by 
time steps back would have to be modified to use the new API

Both cases require that application programmers are aware of the 
underlying problem, which often doesn't seem to be the case. :-(

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 22 Jul 2015 17:36:47 +0200

> But kernels don't step the time back; they insert an extra 23:59:60 second, right?

Unfortunately not right. Kernels *do* step the time back, and that's the 
main reason of most leap second problems.

I've put a presentation together which tries to explain some basics:
https://www.meinbergglobal.com/download/burnicki/Technical%20Aspects%20of%20Leap%20Second%20Propagation%20and%20Evaluation%20-%202015-06-23.pdf

There you can see a log showing how the Linux kernel steps time back at 
the beginning of an inserted leap second.

The time returned from the kernel is binary, seconds since 1970 plus 
fractions of a second. No chance to see a second :60 there.

Only if the binary time stamp is converted to human readable date and 
time the result could be something like "23:59:60". However, the 
CLOCK_REALTIME call just returns the same number for the second twice 
when a leap second is inserted the runtime library yields the same 
broken down time twice, namely "23:59:59".

The runtime library could only get the conversion to :60 right if there 
was some status associated with the time stamp which would flag the 
second occurrence as a leap second. Unfortunately the commonly used API 
calls like clock_gettime(CLOCK_REALTIME) don't provide such status.

> So what are the real problems?

Applications get, and even versions of the Linux kernel itself got 
confused when the time is stepped back to insert a leap second. See:

Linux Kernel Mailing List, "Potential fix for leapsecond caused futex 
related load spikes", 2012-07-01
https://lkml.org/lkml/2012/7/1/27

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Warner Losh <[email-masked]>
Date: Wed, 22 Jul 2015 09:48:30 -0600

> On Jul 22, 2015, at 9:19 AM, Martin Burnicki <[email-masked]> wrote:
> 

Here’s where we disagree, I think. Since this is the default clock, and libraries will rarely know what type of system they are running on, I think expecting programmers to use a different one is a bit too much.

> - either CLOCK_REALTIME had to be modified to return slewed time without a step. In this case applications which get confused by time steps back would work properly, but applications which expect exact UTC time might fail,
> 
> - or a new API could be introduced which returns slewed system time without a step back, but in this case applications which get confused by time steps back would have to be modified to use the new API

I’ve given up on fixing this via an API change. Generally, systems care, not applications, about the system clock. By that I mean when an application knows about the weirdness of leap seconds, generally it is part of a well-architected system that knows about leap seconds. The systems I’m familiar with that are like this assume time is the same for all the applications that are running on the system, since many things are one cooperatively. It also assumes that time is the source for UTC without too many hoops (just the adj_timex niggle around leap seconds). Those systems you want to have a unified view. Those system also use a fair number of libraries. Some of these libraries use posix-threads, which have an unfortunate API (some timeouts are specified as absolute times). This API often isn’t used properly (people use CLOCK_REALTIME rather than CLOCK_MONOTIME because the former is the default). In large systems, it can often be quite difficult to find all such bugs and fix them (and sometimes the timeout is related to something in CLOCK_REALTIME legitimately). There’s already APIs that people can use to get around the problem. However, they don’t use them religiously. More APIs won’t help.[*]

The systems I’ve been involved in bit the bullet and had CLOCK_REALTIME run in UTC to the extent possible, and step back the clock to implement leap seconds. We used CLOCK_MONOTIME for all the crazy short-term timeouts that were around in the system, and had an elaborate system to keep track of leap seconds (which has been re-written about 4 times in the last decade since I left). Here we absolutely did not want skewed time. This system put out the pulses that kept LORAN-C on time, so there was also auditing that had to be in UTC with a special waver for some logging data at the leap second being logged in the wrong second. Skewed time for these would make it hard to correlate failures on leap second day. The logging sources are many different processes running different programs, some of which are third party, others were custom built. They all got their time from CLOCK_REALTIME.

On the other hand, I know of a lot of web servers that just don’t care. They don’t want to have to deal with the odd-ball special case in their logging with leap seconds, or have data spikes around the leap second if it gets counted the same as another second. They have a much more relaxed need for time. Here, just setting the system to skew in one place makes it real easy to paper over this issue by skewing. The small error you’d introduce in the frequency would get spread over all the seconds, making the errors below a few hundred parts per million. Far into the noise.

> Both cases require that application programmers are aware of the underlying problem, which often doesn't seem to be the case. :-(

If application programmers were aware of the problem on a more regular basis, we wouldn’t be having this conversation. :-(

Warner

[*] Think Futurama where the trains crash due to problems on the tracks and the governor orders more trains to be sent in without fixing the tracks…

---

From: brian utterback <[email-masked]>
Date: Wed, 22 Jul 2015 13:24:12 -0400

> - either CLOCK_REALTIME had to be modified to return slewed time
> without a step. In this case applications which get confused by time
> steps back would work properly, but applications which expect exact
> UTC time might fail,
>
> without a step back, but in this case applications which get confused
> by time steps back would have to be modified to use the new API
>

Here's how I see it. As we have discussed, what to do about leap second
is largely a policy decision and depends on the needs of the end
applications and the timescale in use. So, in an ideal world,
applications would get time from the kernel via an API that provides
enough information to make and implement those decisions all in user
space. I presume that the GTAPI likely be that API. Given that we do not
worry about leap seconds in NTP or the kernel and do no special
handling, assuming the correct kernel timescale choice. So, the first
question is, what should that timescale be?

Now of course most applications will not be using the GTAPI, so what
should the standard Unix system time related API's do in our ideal
world? In general we can do anything we want in the system libraries, so
should we do timescale conversions in the libraries, perhaps based on
system policy decisions or environment variables? Or should those
conversions be done in the kernel? I think that this is sort of the
approach of the Linux Right timezones. (does anyone have any links that
discuss the Right timezone implementation?)   If it is in the kernel,
then the kernel needs to know about everything it needs to do the
conversion. If in the user space, the same info needs to be available
there.

This implies that the kernel needs to reliably know about leap seconds,
presumably either from a leap seconds file or from NTP. This also says
that on this hypothetical system NTP never smears, slews or or otherwise
lies about the time as a result of the leap second. In a world full of
these systems, NTP is agnostic to leap seconds, except for the leap
indicator. This implies that NTP should also be using a timescale that
is agnostic to leap seconds. This also says to me that the leap
indicator should be part of an extension file since it is only used once
every coupld of years. Just a thought for NTPv5.

Perhaps there is no backwards compatible, POSIX compliant way to do this
in userspace. In that case, the leap handling needs to be in the kernel
. Obviously it would be a lot easier to implement systemwide policies
rather than application specific ones. Indeed, we might run into
problems with file system timestamp inconsistencies if we do specify
what timescale is needed system wide. In this case, we have a problem.
We want the kernel to have the flexibility to smear, step, stop, etc.,
the clock for leap seconds, but NTP still needs to be able to read and
set the clock without any leap handing. Perhaps a new set/get_clock
CLOCK_NTP type?

Now suppose ntpd is running on a system that is legacy, with standard
kernel and current applications. In that case ntpd needs to handle the
leap second processing. This means that ntpd needs to be able to break
out of it's standard processing and more actively adjust the system
clock, at least for smearing. It also means that ntpd needs to know
exactly what the current offset between the smeared time the kernel is
using and the actual time being served by the other NTP servers. This
shouldn't be a problem since it is ntpd itself that is determining and
applying any offset.

Now, finally, we get to the case where there is a legacy system running
a legacy ntpd. This is the only case where we want the timestamps in the
NTP packet to already reflect some leap second handling policy. In this
case we want an up to date ntpd to be able to recognize that the
timestamps do not reflect actual time and to be able to correct them to
get the real time. There is no way in the current NTPv3 and NTPv4
protocols to flag the timestamps as having been adjusted in a manner
that is propagated through the network. Further, the leap second
indicator should not be set when this is happening since it would be
possible for some systems to end up applying a leap second correction
twice, thus resulting in the worst of all worlds. Using a special REFID
at the server that applies the offset might make it more likely to be
able to diagnose issues, but I am not sure this is worthwhile. Applying
the offset to the root dispersion has the advantage of making the offset
more apparent, but means that for networks with already high root
dispersions the offset might put them over the threshold and stop NTP
from working at all.

So, anyway, there is my first stab at some of the goals and issues.
Perhaps we could put this into a wiki page at ntp.org?

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 22 Jul 2015 17:50:54 +0000

[email-masked] writes:
> So a client that may well have backed off to 1024 sec polling (V3 or
> greater for V4) will see a server announce a leap second. But then
> over a not well specified interval (let's say an hour) effectively
> become a false ticker. The client will only poll three times in this
> period itself, which is not enough for it to start taking action
> anyway.

Yes.  If folks don't have a good understanding of what can happen they
can easily create configurations where they'll screw themselves over.

This means people will need to remember that "leap seconds happen" when
they decide on the value for maxpoll.

> It would make more sense to create a client profile that can act on
> the existing leap second flag and then "smear" time to its host
> system, rather than break the server architecture and propagate the
> wrong time.

I'm not sure what you mean by that paragraph.

H
--
> >> Well, "smear" is not a technical term with clear semantics, so the obvious
>  interpretation is anything that changes the clock frequency from the correct
>  frequency IMHO.
> > I have not seen a technical specification, but I think in the context of th
> is discussion "slew" means that a client adjusts its system time gradually, t
> o apply a leap second or another correction, the slewed system time is sent o
> ut in reply to client requests, and also other applications running on the sl
> ewing machine see the slewed system time.
> > 
> > Contrarily, "smearing a leap second" as I've implemented it for ntpd means 
> that all the internal timing is unchanged, i.e. ntpd passes the leap second w
> arning to the kernel, the kernel steps the time back to insert a leap second,
>  etc. Only the time put into the reply packets on client requests is modified
>  by a "smear offset" which increases from 0 to -1 s over the smear interval, 
> so that the sent time matches UTC again after the end of the leap second.
> > 
> > Martin
> 
> The first coherent description by Markus Kuhn used the term "smoothed leap se
> conds", and I think smoothing is a better description of the operation:
> 
> It is more correct to view UTC as a standard in units of tenth-seconds. This 
> is what it provides formally as an approximation to UT1.
> 
> So another way to look at smoothing / smearing is as spreading that 10 step a
> djustment (in tenth-seconds) out over some period of time. At the nominal NTP
>  slew rate of 0.5 ms per second, each tenth-second step would take 200 second
> s to accomplish. How NTP (or other software) explicitly implements smoothing 
> of leap seconds (e.g., linear versus cosine, etc.) is a matter for discussion
>  before the next leap second. One hopes Martin's prototype generated some goo
> d field data to analyse from the latest leap second.
> 
> Perhaps the word "smear" should be reserved for Google's implementation? A sy
> nonym for smooth is "continuous", and this is the word the ITU has chosen to 
> emphasize as a key feature needed for UTC. Implementing smoothed / smeared le
> ap seconds is one way to achieve this goal.
> 
> Rob Seaman
> Network Time Foundation
> [email-masked]

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 23 Jul 2015 08:19:18 +0200

>>> Warner Losh <[email-masked]> schrieb am 22.07.2015 um 17:05 in Nachricht
<[email-masked]>:

>> On Jul 22, 2015, at 7:12 AM, Ulrich Windl
<[email-masked]> 
> wrote:
>>>> But kernels don't step the time back; they insert an extra 23:59:60
second,
>>>> right?
>> Thanks, I see: The POSIX time structs don't have a leap indicator, and 
> adding it might break some badly designed applications (relying on the size

> of the struct).
> 
> time_t is typically a simple integer of some kind (though some oddball 
> floats exist). There’s no room to add a leap indicator. And the formula
from 

I was thinking of the newer POSIX struct, i.e. struct timespec. time_t is the
very old UNIX time specification.

> the standard for the value of the time_t precludes them entirely. Encoding 
> the leap second using the standard formula gives a time_t value that’s the

> same as the following second. That’s why time steps backwards on kernels
that 
> do leap seconds: they have to repeat some value. Some repeat the last second

That would mean time seems to stop for a second, not step backwards, right?

> of the day (since they want to properly have the leap second be part of same

> day), while others repeat the first second of the next day (since that’s
what 
> naive math from the POSIX standard might[*] imply).

That's the next thing: With leapseconds, a day is one second longer (as it is
when you switch from DST to normal (it's one hour then)). 

> 
> time_t has even changed size on a number of recent open source systems 
> without breaking applications at the source level. At the binary level,
every 
> application on the planet encodes the side of an integer in them. That 
> doesn’t make them badly designed, that’s just how C works. I’d go so
far as 

I can remember complaints in the UNIX magazine around 184 when they said
(years before Y2K) UNIX is broken, because the time will overflow in 2038 or
so. At that time nobody could imagine that there would be 64-bit systems,
obviously. So it's quite normal to use a 64-bit time_t on 64-bit machines
(using a 64-bit ABI).

Nodoby had problems with leap seconds, maybe also because everything was quite
slower then.

> to say that relying on the size of the struct is also not badly designed, 
> since that’s supposed to be a run-time invariant.
> 
> It goes well beyond the size of the integer. The semantics of the value of 
> time_t preclude leap seconds. The semantics of C harden sizes of things at 
> compile time.

Personally I'd consider time_t as obsolete (for new applications).
Interestingly the man page of mktime says abozt tm_sec: "The number of seconds
after the minute, normally in the range
0 to 59, but can be up to 60 to allow for leap seconds.", but it does not say
how to find out.
Similar is true for date's "%S" (second (00..60)).

Regards,
Ulrich

> 
> Warner
> 
> [*] I said might here, since blindly applying the formula does give this 
> result. Yet other parts of POSIX are quite emphatic that leap seconds are 
> outside the scope of things that are standardized. The whole issue of leap 
> seconds is quite contentious within POSIX with one side eventually kicking 
> out the other and winding up with the infamous ‘double leap second’
value of 
> 61 sneaking into old versions of the standard.

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 23 Jul 2015 08:50:58 +0200

>>> Warner Losh <[email-masked]> schrieb am 22.07.2015 um 17:48 in Nachricht
<[email-masked]>:

>> On Jul 22, 2015, at 9:19 AM, Martin Burnicki <[email-masked]>

> Here’s where we disagree, I think. Since this is the default clock, and 
> libraries will rarely know what type of system they are running on, I think

> expecting programmers to use a different one is a bit too much.

Expecting the programmers to request what they want is not too much IMHO.
I guess the real problem is that programmers happily mix the concepts of "wall
time" and "elapsed time". Few years ago I also fell into that pit assuming that
every day of the year has 24 hours, when I realized that one day in spring only
has 23 hours, while another in autumn has 25 hours...

>> - either CLOCK_REALTIME had to be modified to return slewed time without a

> step. In this case applications which get confused by time steps back would

> work properly, but applications which expect exact UTC time might fail,

With slewing time you can make the best of the current interface, but the true
thing would be not to slew the time and fix the interface instead. Of course
the first solution is closer to reach.

>> 
>> - or a new API could be introduced which returns slewed system time without
a 
> step back, but in this case applications which get confused by time steps 
> back would have to be modified to use the new API
> 
> I’ve given up on fixing this via an API change. Generally, systems care,
not 
> applications, about the system clock. By that I mean when an application 
> knows about the weirdness of leap seconds, generally it is part of a 
> well-architected system that knows about leap seconds. The systems I’m 
> familiar with that are like this assume time is the same for all the 
> applications that are running on the system, since many things are one 
> cooperatively. It also assumes that time is the source for UTC without too 
> many hoops (just the adj_timex niggle around leap seconds). Those systems
you 
> want to have a unified view. Those system also use a fair number of 
> libraries. Some of these libraries use posix-threads, which have an 
> unfortunate API (some timeouts are specified as absolute times). This API 
> often isn’t used properly (people use CLOCK_REALTIME rather than 
> CLOCK_MONOTIME because the former is the default). In large systems, it can

> often be quite difficult to find all such bugs and fix them (and sometimes 
> the timeout is related to something in CLOCK_REALTIME legitimately).
There’s 
> already APIs that people can use to get around the problem. However, they 
> don’t use them religiously. More APIs won’t help.[*]

As a first step it would be quite helpful to _document_ the issues with the
existing interfaces and leap seconds. Maybe then more programmers become aware,
and maybe there will be some pressure for revised or new interfaces. So the
first step in fixing a problem is to become aware of it...

> 
> The systems I’ve been involved in bit the bullet and had CLOCK_REALTIME
run 
> in UTC to the extent possible, and step back the clock to implement leap 
> seconds. We used CLOCK_MONOTIME for all the crazy short-term timeouts that 
> were around in the system, and had an elaborate system to keep track of leap

> seconds (which has been re-written about 4 times in the last decade since I

> left). Here we absolutely did not want skewed time. This system put out the

> pulses that kept LORAN-C on time, so there was also auditing that had to be
in 
> UTC with a special waver for some logging data at the leap second being 
> logged in the wrong second. Skewed time for these would make it hard to 
> correlate failures on leap second day. The logging sources are many
different 
> processes running different programs, some of which are third party, others

> were custom built. They all got their time from CLOCK_REALTIME.

I wonder whether the would could agree to slew the time from leap second to
leap second. So probably nobody will ever notice, but that's hard to do in the
reference clocks.

> 
> On the other hand, I know of a lot of web servers that just don’t care.
They 
> don’t want to have to deal with the odd-ball special case in their logging

> with leap seconds, or have data spikes around the leap second if it gets 
> counted the same as another second. They have a much more relaxed need for 
> time. Here, just setting the system to skew in one place makes it real easy

> to paper over this issue by skewing. The small error you’d introduce in
the 
> frequency would get spread over all the seconds, making the errors below a 
> few hundred parts per million. Far into the noise.
> 
>> Both cases require that application programmers are aware of the underlying

> problem, which often doesn't seem to be the case. :-(
> 
> If application programmers were aware of the problem on a more regular 
> basis, we wouldn’t be having this conversation. :-(

Yes, see above.

Regards,
Ulrich

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 23 Jul 2015 09:06:10 +0200

>>> Harlan Stenn <[email-masked]> schrieb am 22.07.2015 um 19:50 in Nachricht
>> period itself, which is not enough for it to start taking action

So you are talking about a falseticker regarding to the leap indicator? I'm not sure, but I think the leap announcement is accepted from the system peer whatever other candidates say, right? (RFC 5905 seems to confirm my opinion)
If so, you can't speak of a "falseticker" regarding to the leap announcement. Maybe a "leap consensus" should be required.

> they decide on the value for maxpoll.

I think the problem is in the specs, not in the configuration.

[...]

Regards,
Ulrich

---

From: Warner Losh <[email-masked]>
Date: Thu, 23 Jul 2015 12:02:59 -0600

> On Jul 23, 2015, at 12:19 AM, Ulrich Windl <[email-masked]> wrote:
> 

The tv_sec field is of type time_t. But having a ‘bit’ that says ‘this is a leap second’ is about the lamest API you can have. tv_sec gets copied all over the place, so having to also copy this bit would be burdensome. Leap seconds simply aren’t part of the time_t specification, and trying to shoe-horn them in will lead to trouble.

> That would mean time seems to stop for a second, not step backwards, right?

No. You can make cases for all three strategies, and they are all POSIX compliant because POSIX doesn’t specify anything about what happens during the leap second.

> when you switch from DST to normal (it's one hour then)).

Not quite. With DST the day isn’t one hour longer or shorter as recommend by the kernel. All that nonsense is handled in user land. With leap seconds, that’s typically done in the kernel, which leads to many different types of crazy, and many different supposed invariants that are violated. POSIX forced your hand because it can only honor so many of the invariants and your implementation is then forced to pick which one(s) it wants to sacrifice so that other, more important ones can be retained.

> Nodoby had problems with leap seconds, maybe also because everything was quite

Nobody had GPS receivers then….

> Similar is true for date's "%S" (second (00..60)).

Yes. Those are flaws in the standard. It says it ‘can be’ this, but then is silent about how you can ensure that it is.

However, time_t is far from obsolete. There is no replacement for it. Doing math with ‘broken down time’ like the type mtkime() uses is cumbersome at best. For all its flaws, time_t math is very straight forward (which I believe POSIX committee members have publicly stated was more important than leap seconds).

Warner

---

From: Warner Losh <[email-masked]>
Date: Thu, 23 Jul 2015 12:05:09 -0600

> On Jul 23, 2015, at 12:50 AM, Ulrich Windl <[email-masked]> wrote:

These issue have been extensively documented. Awareness isn’t going to fix this. People have tried for about two decades to get people educated on these issues and it hasn’t helped. And these interfaces are so deeply engrained that it is unlikely they will ever be changed or phased out.

Warner

---

From: Brian Utterback <[email-masked]>
Date: Thu, 23 Jul 2015 14:47:52 -0400

> These issue have been extensively documented. Awareness isn’t going to fix this. People have tried for about two decades to get people educated on these issues and it hasn’t helped. And these inter…

I disagree. The interfaces have to be provided that can deal with the 
needs of the applications correctly, with as much of the complexity 
abstracted below the interface and the correct usage of the interfaces 
well documented. There are steps that need to be taken before we can 
take the high road and say that an application is buggy and needs to be 
fixed. I have never seen any docs on handling leap seconds for instance, 
only about the effect a leap second might have for application that 
didn't take them into account.

---

From: Steve Allen <[email-masked]>
Date: Thu, 23 Jul 2015 12:36:24 -0700

On Thu 2015-07-23T14:47:52 -0400, Brian Utterback hath writ:
> I have never seen any docs on
> handling leap seconds for instance, only about the effect a leap
> second might have for application that didn't take them into
> account.

For the most recent leap second the US Deparment of Homeland Security
and the Coast Guard released this best practices document
http://www.navcen.uscg.gov/pdf/cgsic/Leap_Second_Best_Practices_20150526_Intrl_Version.pdf
It avoids any language which might be interpreted as allowing a "smear".

In contrast, Google, Amazon, Microsoft and most of the financial
industry implemented a "smear".
http://googlecloudplatform.blogspot.com/2015/05/Got-a-second-A-leap-second-that-is-Be-ready-for-June-30th.html
https://aws.amazon.com/blogs/aws/look-before-you-leap-the-coming-leap-second-and-aws/
http://www.theregister.co.uk/2015/05/29/windows_azure_second_out_of_sync/
https://mt.fia.org/events/fia-market-technology-division-webinar-leap-second-event-preparedness

The upcoming ITU-R WRC-15 which will again consider leap seconds in
the radio broadcast time scale.  The most recent view into the
proposals on the table shows methods A1, A2, B, C1, C2, and D which
are not in agreeement with each other:
http://www.acma.gov.au/Industry/Spectrum/Spectrum-planning/International-planning-ITU-and-other-international-planning-bodies/wrc-15-agenda-item-114

Methods A and B are from nations whose delegations assert that some
change must be made, and method D is the newest method from a group of
delegations who assert that no change should be made because the leap
second evidence presented to the ITU-R during the past 15 years is
inconclusive.

This is a failure to agree that has been ongoing for 50 years.  The
NTPWG will have to work in a focussed manner to create a technical
basis for consensus.