Subject: Re: [art] Predictable Internet Time

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 14:40:07 -0700

Moving this to ART...

On 3/28/2017 10:39 AM, Nico Williams wrote:
> On Tue, Jan 03, 2017 at 06:35:03PM +0000, Stewart Bryant wrote:
>> Yes, the system should use a leap-second-free constant-duration-seconds time
>> for everything. It is only humans that need the variable jumpy version of
>> time presented to them, and that is a UI issue.
> I don't think that's quite right.
>
> Unix time is canonically a count of seconds since an epoch and admits no
> leap seconds. 

Unix time also uses a completely different definition of a "second" than
UTC.

There is no simple conversion between Unix time and UTC. Conversion
requires accurate assessment of the local Unix clock vs SI to determine
a rate conversion as well as knowing the table of leap seconds since the
Unix epoch.

> ...
> It's all about data typing. 

That is certainly one problem, but not the only one.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 14:43:31 -0700

On 3/28/2017 12:16 PM, Nico Williams wrote:
>> With a suitable definition, PIT could create a condition in which it would
>> only take a decision by one major government to force a change on the
>> astronomers. The commercial advantages of PIT over UTC are obvious - fewer
>> things are going to break for no good reason. That is an argument that
>> every politician is willing to listen to.
> Do we have this much influence?  If we do, why not just... lobby for
> fixing UTC and creating an TAC (tempt astronomique coordiné) to serve,
> for astronomers, the function that UTC serves today.
We don't need TAC; we already have UT. IERS already broadcasts the
UTC-UT1 delta.

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 17:40:42 -0500

> Unix time also uses a completely different definition of a "second" than

Did I.. say otherwise?

> a rate conversion as well as knowing the table of leap seconds since the

You're complicating things.  Forget the local clock.  To convert a time_t
value to UTC requires just a list of leap seconds (and some arithmetic).

There's no need to assess the quality of the local clock or anything of
the sort.

> That is certainly one problem, but not the only one.

It's really the main one, IMO.  In any case, smearing seems very wrong:
you end up with more problems because you go from roughly two kinds of
time (UTC vs TAI-ish) to three kinds of time (UTC vs TAI-ish vs the new
thing).  If people had a hard time interoperating with two kinds of
time, imagine how it would be with THREE kinds of time.  And if the
smearing formula ever needs updating, then we'd be in trouble.  (Earth's
rotation normally _slows_ over time, but the 2004 earthquake _sped up_
Earth's rotation.  A few big ones and we might need negative leap
seconds.  PIT's formula can't predict these events.)

Just specify, in each protocol, the use of TAI (x)or UTC.  Done.

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 16:02:46 -0700

>> Unix time also uses a completely different definition of a "second" than
Yes, below....
> value to UTC requires just a list of leap seconds (and some arithmetic).
time_t does not access a SI reference or frankly any other stable reference.

If you don't have a stable time unit, accurate conversion isn't possible.

If you do have a stable time unit, you need the ratio of that time unit
to SI.

> Just specify, in each protocol, the use of TAI (x)or UTC.  Done.
I agree.

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 19:06:02 -0500

> On 3/28/2017 3:40 PM, Nico Williams wrote:

You're picking non-nits.

> time_t does not access a SI reference or frankly any other stable reference.

The Open Group man pages I'm looking at don't reference SI, but so
bloody what.  If it says "seconds", it means "seconds".

> If you don't have a stable time unit, accurate conversion isn't possible.

Sure, but POSIX time clearly uses the only definition of seconds that
matters.

Also, time is relative.  These seconds are seconds at mean sea level on
Earth, but they might vary nonetheless.  At some point precision is not
so interesting, but knowing 1490745768 is 2017-03-29T00:02:48Z, that's
kinda important, and that will be true in any local frame of reference,
even in space, far from Earth, and under large accelerations.  Sure, two
such observers won't agree as to current time because of relativistic
effects, but they will agree that 1490745768 is 2017-03-29T00:02:48Z.

> If you do have a stable time unit, you need the ratio of that time unit

It's very safe to assume it's 1.  Anything else would be insane.

> > Just specify, in each protocol, the use of TAI (x)or UTC.  Done.

Isn't that the important bit of information here?

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 17:15:58 -0700

> bloody what.  If it says "seconds", it means "seconds".
That's the trouble - seconds in SI are not seconds defined as 86400/day.
If the units don't match,  then a conversion needs to be included. If
the units aren't known, you can't convert accurately.

> effects, but they will agree that 1490745768 is 2017-03-29T00:02:48Z.
2017 is 1.4e9 seconds after the Unix epoch. If the Unix and SI "seconds"
unit is off by one part in a (US) billion, then you're off by 1.5 seconds.

> It's very safe to assume it's 1.  Anything else would be insane.
s/insane/undefined/

That's the problem. I appreciate it would be nice if there were a
definition (as a start), and if the defined unit were SI, but it isn't,
AFAICT. If you have a ref to the contrary, I'd be more than glad to
include it and correct the doc accordingly.

> Isn't that the important bit of information here?
To you and me, perhaps, but there are others on this thread that are
misled by what POSIX is and how easy/hard it is to convert to TAI/UTC...

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 20:58:12 -0500

> the units aren't known, you can't convert accurately.

POSIX does NOT define seconds in terms of days!  It does the opposite:

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

    How any changes to the value of seconds since the Epoch are made to
    align to a desired relationship with the current actual time is
    implementation-defined. As represented in seconds since the Epoch,
    each and every day shall be accounted for by exactly 86400 seconds. 

There is NO reference to solar or sidereal days or any other
astronomical definition of day.

Unlike as with days, there is no multiplicity of generally accepted
definitions of seconds, so there is no support for the idea that this
text defines seconds in terms of days.

The second sentence defines days in terms of seconds, not the other way
around.

The first sentence is a bit abstruse, but no matter, because in
_practice_ on all systems I know of that claim to implement POSIX, or
which don't but which are Unix or Unix-like systems, seconds since the
epoch differs from UTC *only* by leap seconds -- just like TAI.

Also, POSIX does refer to UTC:

    A value that approximates the number of seconds that have elapsed
    since the Epoch. A Coordinated Universal Time name (specified in
    terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days
    since January 1 of the year (tm_yday), and calendar year minus 1900
    (tm_year)) is related to a time represented as seconds since the
    Epoch, according to the expression below.

But as others have pointed out, that is either a lie or an error: POSIX
simply does not and cannot account for leap seconds in a numeric type
representing seconds since an epoch.  Perhaps we can say that the
reference to UTC was meant to be to TAI.  Either way the definition of
seconds can only plausibly be SI.

The text is sufficiently ambiguous (because of its errors, for example)
that you could remotely plausibly make these claims that you do, but
practice does not bear them out.

> include it and correct the doc accordingly.

POSIX seconds are not defined, indeed, but they can only be taken to be
SI seconds -- anything else would be sheer madness and, anyways, not
supported by actual practice.  I know of no POSIX or POSIX-ish system
where this is not true.

Running code, _running code_.

> misled by what POSIX is and how easy/hard it is to convert to TAI/UTC...

Maybe, but so are you.  Not that that is particularly important to
making a successful argument against smearing, but that such errors are
distracting.

The obvious argument against smeared time is that we already have _two_
time standards, and we don't need a third.  All we need is to carefully
pick one or the other in each specification, and preferably TAI time in
all cases.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 21:58:13 -0500

> POSIX does NOT define seconds in terms of days!  It does the opposite:

Although I can see how to waterboard the spec to get either answer.  If
it were right about POSIX time being UTC (which it's not in practice),
then you'd be quite right about it defining seconds in terms of days...

Would that make POSIX a smeared time standard?  Sure seems that way!

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 07:35:06 -0700

Hi, Nico,

> astronomical definition of day.
OK - we agree that it defines "day" = 86400 "seconds".

It should be clear that it also never defines either "day" or "seconds".

"Seconds" can't be SI units unless there's a tiny cesium clock inside my
PC. "day" can't be apparent or mean solar time because it doesn't have a
sextant either.

> epoch differs from UTC *only* by leap seconds -- just like TAI.
The text only states that the POSIX clock *started* with a known offset
to TAI and that the POSIX clock won't jump with leap seconds.

It never states anything else...

> supported by actual practice.  I know of no POSIX or POSIX-ish system

I know of no POSIX system with a cesium atomic clock.

The issue is that POSIX really runs off of whatever local source of time
passage is available, and can (and does) drift from TAI on *each*
system. That drift can't be determined until your system tries to sync
with an external source (e.g., via NTP).

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 07:35:59 -0700

> Would that make POSIX a smeared time standard?  Sure seems that way!
POSIX never claims that its "days" sync to UTC, UT, or TAI days. As
such, there's no smearing assumed.

Joe

---

From: Philip Homburg <[email-masked]>
Date: Wed, 29 Mar 2017 16:53:01 +0200

>with an external source (e.g., via NTP).

To be completely pedantic, a single cesium clock will also drift from TAI/UTC.

There is no such concept as being in sync with TAI without external
corrections.

To be even more pedantic, of course that depends on the desired accuracy.
A cesium clock could easily stay within one hour from TAI for a period of a 100
years. A random PC with a quarz crystal at room temperature would have a hard
time doing that.

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 07:57:30 -0700

> There is no such concept as being in sync with TAI without external

Yes - I got email off-list about this and need to update the doc to
remind us that TAI is actually determined post-facto, though its current
value is approximated.

> years. A random PC with a quarz crystal at room temperature would have a hard

Right - I will update the doc accordingly. AFAICT, POSIX time implies
(but never states) that it uses a local reference clock to emulate TAI.

Joe

---

From: Philip Homburg <[email-masked]>
Date: Wed, 29 Mar 2017 17:13:07 +0200

>(but never states) that it uses a local reference clock to emulate TAI.

Not TAI, UTC.

Every time_t value has a specific interpretation as a UTC timestamp. Obviously,
without correction a free running clock will go wrong with leap seconds. But
that doesn't change that it is some approximation of UTC and not TAI.

By and large, current Unix systems have no idea about TAI. There are no 
(commonly available) APIs that give you TAI.

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 08:21:15 -0700

>> (but never states) that it uses a local reference clock to emulate TAI.

POSIX doesn't include leaps. It can't track UTC.

Maybe we're talking across things -

POSIX defines only one internal clock - the Unix one that starts at the
Unix epoch and counts up.

time_t is supposed to return an integer of this count

If you're referring to a time_t structure that indicates days, weeks,
months, etc., you're talking about a *conversion* that *approximates*
UTC, but really basically converts the Unix clock as close as it can to
TAI and ignores the leaps.

> Every time_t value has a specific interpretation as a UTC timestamp.
There is an intended correlation between Unix dates and UTC dates, but
it isn't exact - the equation is indicated as approximate.

>  Obviously,
POSIX says it doesn't track leaps, which means (at best) it emulates
TAI, not UTC. Even if the documentation of the time_t structure and
commands indicate otherwise...

> (commonly available) APIs that give you TAI.

The time_t struct is a conversion of the Unix count since epoch into a
"UTC-like" format, but it really is much more an emulation of TAI than UTC.

But it *is* really neither, because it isn't sync'd to anything.

Joe

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Mar 2017 10:30:34 -0500

> >I know of no POSIX system with a cesium atomic clock.

The preceding bit about POSIX seconds being unable to be SI seconds
without an atomic clock speaks for itself.  Joe may be the only one who
believes that.

> >with an external source (e.g., via NTP).

Which... most do.

> There is no such concept as being in sync with TAI without external

To be even more pedantic that may not be meaningfully possible or useful
or desirable due to being in an irregular orbit around Earth (e.g.,
frequently using thrusters to adjust it) or not in orbit around Earth,
and not on Earth.

This is not a helpful distraction.  It's wasting everyone else's time.
Also yours and mine.

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 08:34:18 -0700

>>> with an external source (e.g., via NTP).
Then you're reporting NTP time, not Unix time.

And that's when you're clock jumps - because you're looking at UTC, not
the Unix clock anymore.

Joe

---

From: Philip Homburg <[email-masked]>
Date: Wed, 29 Mar 2017 18:11:32 +0200

>it isn't exact - the equation is indicated as approximate.

Quoting the opengroup docs, and this is the last thing I will say
about POSIX time:

"4.16 Seconds Since the Epoch
"
"A value that approximates the number of seconds that have elapsed since the Epoch. A Coordinated Universal Time name (specified in terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is related to a time represented as seconds since the Epoch, according to the expression below.
"
"If the year is <1970 or the value is negative, the relationship is undefined. If the year is >=1970 and the value is non-negative, the value is related to a Coordinated Universal Time name according to the C-language expression, where tm_sec, tm_min, tm_hour, tm_yday, and tm_year are all integer types:
"
"tm_sec + tm_min*60 + tm_hour*3600 + tm_yday*86400 +
"    (tm_year-70)*31536000 + ((tm_year-69)/4)*86400 -
"    ((tm_year-1)/100)*86400 + ((tm_year+299)/400)*86400

Feel free to define POSIX time as you wish. This what the opengroup says
about it.

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 09:16:21 -0700

> Quoting the opengroup docs, and this is the last thing I will say

We've all been referring to this.

The problem is that POSIX never defines second, and that this entire
paragraph starts with "a value that approximates".
It also refers to "a Coordinated Universal Time name", that's a data
structure, not an assurance that seconds since epoch actually
synchronizes to UTC.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 09:43:13 -0700

Hi, all,

So I have a set of things to update so far. Please let me know if I
missed something. I will have an update out by the end of this week.

I'm hoping this update can help us converge...

Joe
---------

- clarify TAI as a post-facto average
- clarify prev definition of astronomical second as instantaneous:

1/31556925.9747 of the tropical year as of the instant
1900 January 0 at 12:00:00 Ephemeris Time.

- clarify UT1 as a a measure of the orientation angle of the earth upon
which international agreement bases the notion of calendar days.
- consider cleaning up the UT definitions, noting mostly that UT1 is the
only one in use (and not getting into the detail of UT0 and UT2)
- clean up the POSIX definition (it's not solar; more like a local clock)
- this is all at mean sea level (we're ignoring relativistic effects)
- reminder that the POSIX "equation" is approximate
- a light discussion about implementation issues (e.g., struct timeval
vs struct tm vs. ISO 8601)
- fix NTP epoch to 1900-01-01
- clarify the two solar times as "apparent" vs "mean"
- reminder that timers either end on a date or specify an interval
- reminder that future times in UTC are indeterminate
- UTC is known only into some future (e.g., a few months) that you know
about scheduled leaps
- earth rotation, not orbit, causes most of solar time variation

One other thing that came up is the difference between "POSIX time"
(previously called Unix time) and the POSIX time API. The API can access
a variety of time scales, including UTC (e.g., when NTP is running), an
emulation of UTC, an emulation of Unix time, or direct access to a
monotonic counter (which may or may not be reset on reboot). I think
this is Nico's issue with data types, and it's worth clarifying.

-----