Subject: [Ntp] Smearing

From: Hal Murray <[email-masked]>
Date: Fri, 15 Oct 2021 14:02:50 -0700

> 2. Introduce some means to communicate the existence of smearing, and
> either standardising the smearing cadence or have some way of signalling
> it. Clients may be able to mix same-cadence smearing servers but cannot mix
> between smearing and non-smearing. This only legitimises the status quo
> bodge and makes it explicit. 

The problem with smearing is that the whole idea is to hide it from the 
client.  Currently, there is no way for a user program to figure out if the 
system it is running on is smearing or not.  That includes ntpd.

I think the protocol should use TAI and make available the leap info.

That means an NTP client will need to know if it should smear.

Plan A is to put that info in a flag in /etc/smear or such so that other 
programs can also use it, perhaps to unsmear or perhaps just to print a 
warning message when log files do/don't smear.

Plan B would be to put that info into the kernel.  There are spare slots in 
struct timex and struct ntptimeval.  Then a flag in ntp.conf could tell ntpd 
to smear and it could pass that to the kernel.

--------

Does a smearing client need to know anything more from a server than the 
current TAI offset and the time of the next leap?

The next leap info is big enough that I don't want it in the basic packet format.  If we put it in an extension, dumb programs will ask for it every time.  Do we want to support that?  Or perhaps, do we want to actively discourage that?

How about a new mode for inquiring about things that only change infrequently?  Is there anything besides leap info that the server might want to distribute?

--------

That brings up another issue that I haven't seen documented.  (Maybe I just forgot.)

NTPv4 has version and mode fields.  When decoding a packet, which one do I look at first?

If we look at the version field first, then we can redefine the mode field.  Or move it.

-- 
These are my opinions.  I hate spam.

---

From: Tony Finch <[email-masked]>
Date: Tue, 19 Oct 2021 16:47:56 +0100

> to smear and it could pass that to the kernel.

Both plans don't answer the question of how the client gets the correct
configuration. Smearing is (typically) a feature of a data centre
environment and each client implicitly gets a time that matches its
neighbours by using the local NTP servers.

When there's a mixture of NTP4 and NTP5 clients, how can you be sure that
they get the same time? I think an NTP5 server should have some way to
say, "if you were talking NTP4 to me I would give you a smeared time", so
that an NTP5 client can match its neighbours.

> --------
>
> The next leap info is big enough that I don't want it in the basic
> packet format.

It doesn't need to be more than 4 bytes.

According to TF.460 (the definition of UTC) leap seconds can only happen
at the end of a month, so the leap indicator only needs to count months.

Leap seconds will no longer work when the earth is turning slowly enough
that we need one at the end of every month; we expect that to happen
before the year 4200, at which point DTAI will be about 4 hours.

https://www.ucolick.org/~sla/leapsecs/dutc.html

If you allow 15 bits for the count of months since 1972, that lasts until
after the year 4700. 14 bits for DTAI is over 4.5 hours; it might be
wise to add a sign bit in case things get really weird.

That leaves 2 bits spare which is just enough for the leap direction.
(Though it would be handy to have another bit to indicate smearing.)

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  https://dotat.at/
protect and enlarge the conditions of liberty and social justice

---

From: Danny Mayer <[email-masked]>
Date: Tue, 19 Oct 2021 11:57:41 -0400

> (Though it would be handy to have another bit to indicate smearing.)

I agree. My proposal includes a month count and a leapsecond count. A 
zero for both month count and leapsecond count indicates that the values 
are unknown. It also indicates if there is a leapsmear in progress, the 
type of smearing done (there are a number of different ones) by 
including a smearing ID. The client can then choose whether or not to 
follow that smearing type.

Danny

---

From: Tony Finch <[email-masked]>
Date: Tue, 19 Oct 2021 17:15:16 +0100

Danny Mayer <[email-masked]> wrote:
>
> It also indicates if there is a leapsmear in progress, the type of smearing
> done (there are a number of different ones) by including a smearing ID.

Are there good reasons for needing more than one kind of smear?

As I understand it, the most prominent leap smear deployments have
converged on:

  * 24h smear, not too long but long enough that NTP clients will notice
    the rate change even with a 1024 second polling interval, and the rate
    change is small enough that NTP clients are willing and able to follow it

  * smear from midday to midday around the leap second, to keep the phase
    error below 0.5 seconds

  * use a linear smear, because NTP clients are better at following
    constant rates (a cosine or other smoothing means the rate changes
    continuously during the smear, which NTP does not handle so well)

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  https://dotat.at/
Southeast Iceland: Northeasterly 7 to severe gale 9, occasionally
storm 10 in west. Very rough or high, occasionally rough at first in
southeast and later in northwest. Rain then wintry showers. Poor.

---

From: Danny Mayer <[email-masked]>
Date: Tue, 19 Oct 2021 12:40:17 -0400

>      continuously during the smear, which NTP does not handle so well)

You've just outlined the ones we know about. A client may want to follow 
one or the other or none. That's a policy decision so let the client 
make it. That's why embedding a smearing ID in the packet makes sense.

Danny

---

From: Tony Finch <[email-masked]>
Date: Tue, 19 Oct 2021 17:53:54 +0100

Danny Mayer <[email-masked]> wrote:
>
> You've just outlined the ones we know about. A client may want to follow one
> or the other or none. That's a policy decision so let the client make it.
> That's why embedding a smearing ID in the packet makes sense.

Yes, I understand that :-) but I think that having more than one kind of
smearing is a bad idea (even one kind of smearing is arguably too many...)
so maybe it would be slightly better to standardize only the consensus
least bad version of leap smear.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  https://dotat.at/
Fitzroy, Sole: Southwesterly 4 to 6, veering westerly or northwesterly
6 to gale 8, occasionally severe gale 9 later except in south Fitzroy.
Moderate or rough becoming rough or very rough, then high later in
north Fitzroy. Rain or thundery showers. Moderate or good,
occasionally poor.

---

From: Danny Mayer <[email-masked]>
Date: Tue, 19 Oct 2021 12:58:43 -0400

> least bad version of leap smear.

I have a feeling consensus on this will prove elusive! :(

Danny

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 20 Oct 2021 09:25:17 +0200

>     change is small enough that NTP clients are willing and able to follow it

If there are stricter requirement on maximum offset between different
clients, a longer interval might be needed.
>   * smear from midday to midday around the leap second, to keep the phase

A disadvantage of this approach is that it relies on the leap second
to be known at least 12h before the leap. With some reference clocks
it's known only much closer to the leap and it takes time to be
distributed by NTP for a stratum >=2 server.
>     continuously during the smear, which NTP does not handle so well)

A linear smear is easy to work with for humans, but not for NTP. A
cosine smear, or even better a quadratic smear, reduces the maximum
time error of the client's clock as it can more closely follow the
frequency offset.

-- 
Miroslav Lichvar

---

From: Tony Finch <[email-masked]>
Date: Wed, 20 Oct 2021 20:39:23 +0100

> clients, a longer interval might be needed.

I don't think anyone has implemented a longer smear?

> distributed by NTP for a stratum >=2 server.

The history of bugs in this area suggests that an NTP implementation
should not blindly trust a reference clock to indicate a leap second
correctly. The NTP server should have a copy of the leap second table,
e.g. from the tz database, perhaps cross-checked from multiple places, so
it knows months in advance when a leap second is due. Then it can use any
smear profile, and verify whether the leap is being handled correctly by
each of its time sources. And higher-stratum servers can and should do
the same.

> time error of the client's clock as it can more closely follow the

Ah, I thought Google had said their experience was the other way round,
but I remembered wrong. Their web page doesn't explain why they switched
from cosine to linear. (Maybe they wanted the kernel to be able to
unsmear?)

https://developers.google.com/time/smear)

Meinberg has a nice comparison at the link below, though it's missing a
comparison for clients with variable polling intervals and a 24 hour
smear.

https://kb.meinbergglobal.com/kb/time_sync/ntp/leap_second_smearing/ntp_leap_smearing_test_results

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  https://dotat.at/
Tyne, Dogger, Fisher: West or southwest 4 to 6 veering northwest 6 to
gale 8, increasing severe gale 9 at times. In Fisher moderate or
rough, becoming very rough or high later, elsewhere slight or moderate
becoming rough or very rough. Rain or squally thundery showers.
Moderate or good, occasionally poor.