Subject: [TICTOC] comments on draft-stenn-ntp-suggest-refid-00

From: Sharon Goldberg <[email-masked]>
Date: Tue, 5 Apr 2016 14:28:07 -0400

Dear WG,

To follow up on my comments on draft-stenn-ntp-suggest-refid-00 at the
IETF'95 WG meeting just now. The current draft requires the use of an
extension field.  I believe the goals of the draft can be accomplished
without using an extension field, in a backwards compatible fashion.

The goal of the draft is to limit the information exposed by the REFID
while still preserving robustness to "length-1" timing loops where system A
takes time from system B, but system B takes time from system A.
This proposal allow system A to limit the info it leaks in its refID,
without harming any of its legacy clients.

Suppose system A is taking time from system B. Then there are two cases:
1) If A gets a time query from system B, A puts the IP of B in the refID of
its response. This way, even a legacy B can tell it cannot take time from A
because this would cause a timing loop.

2) If A gets a time query from system C, A puts a "nonsense" value in its
refID.  Even a legacy C can see that its IP is not in the refID, and so it
is allowed to take time from A.

One question is what this "nonsense" value should be.  I think it should be
a fixed value. For example 0.0.0.0.  We would not want a randomly-chosen
value since this might collide with actual IP addresses on the network.

Thanks,
Sharon

-- 
Sharon Goldberg
Computer Science, Boston University
http://www.cs.bu.edu/~goldbe

---

From: Sharon Goldberg <[email-masked]>
Date: Tue, 5 Apr 2016 14:42:27 -0400

All, just to summarize the conversation on meetcho just now. I think  that
this proposal would also solve the problem hashed IPv6 addresses colliding
with IPv4 addresses on the same network, which is addressed
in draft-stenn-ntp-ipv6-refid-hash-00.

Why? Because the only time a server will put an IP address in the refID
field is when it is responding to a query from *that specific IP*.  (Else,
it puts the "nonsense" value in the refID.)  So even if a collision
happens, it should not matter, since the only party to see the collision is
the IP that sent the query.

Sharon

On Tue, Apr 5, 2016 at 2:28 PM, Sharon Goldberg <[email-masked]> wrote:

> 1) If A gets a time query from system B, A puts the IP of B in the refID
> of its response. This way, even a legacy B can tell it cannot take time
> from A because this would cause a timing loop.
>
> One question is what this "nonsense" value should be.  I think it should
> be a fixed value. For example 0.0.0.0.  We would not want a randomly-chosen

---

From: Danny Mayer <[email-masked]>
Date: Tue, 5 Apr 2016 15:00:26 -0400

On 4/5/2016 2:28 PM, Sharon Goldberg wrote:
> extension field.Â  I believe the goals of the draft can be accomplished
> while still preserving robustness to "length-1" timing loops where
> system A takes time from system B, but system B takes time from system A. Â 
> without harming any of its legacy clients.Â 
> 
> 2) If A gets a time query from system C, A puts a "nonsense" value in
> its refID.Â  Even a legacy C can see that its IP is not in the refID,
> and so it is allowed to take time from A.Â 
> 
> One question is what this "nonsense" value should be.Â  I think it
> should be a fixed value. For example 0.0.0.0.Â  We would not want a
> randomly-chosen value since this might collide with actual IP addresses
> on the network.
> 
> Thanks,
> Sharon

The problem that I was alluding to in the jabber room is this. I'll keep
the problem simple and to one example.

A takes its time from B over an IPv4 address. A then gets a time query
from C over IPv6. How does it know if B and C are the same system or
different system? A has no information about that.

Is this clearer?

Danny

---

From: Danny Mayer <[email-masked]>
Date: Tue, 5 Apr 2016 15:01:02 -0400

Is this clearer?

Danny

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 05 Apr 2016 19:12:16 +0000

> value since this might collide with actual IP addresses on the network.

I think your proposal will usually work.

There is a potential problem in that A does not have complete
information about what B and C are doing.  It may be that B and C are
effectively the same machine/source of time, and that B and C would look
at the REFID returned by A and decide that there is a timing loop if A
is getting its time from B and C (which is also effectively B) is
considering using A for time.

If we're going for a strictly clean paradigm here we need to note this.

H

---

From: Sharon Goldberg <[email-masked]>
Date: Tue, 5 Apr 2016 15:21:16 -0400

On Tue, Apr 5, 2016 at 3:00 PM, Danny Mayer <[email-masked]> wrote:

> Yes, it's clear. How does the current ntpd implementation deal with this
problem?

Sharon

---

From: Danny Mayer <[email-masked]>
Date: Tue, 5 Apr 2016 15:28:00 -0400

On 4/5/2016 3:21 PM, Sharon Goldberg wrote:
> On Tue, Apr 5, 2016 at 3:00 PM, Danny Mayer <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     > extension field.Ã‚Â  I believe the goals of the draft can be
>     accomplished
>     > system A takes time from system B, but system B takes time from
>     system A. Ã‚
>     > without harming any of its legacy clients.Ã‚
>     >
>     > its refID.Ã‚Â  Even a legacy C can see that its IP is not in the
>     refID,
>     > and so it is allowed to take time from A.Ã‚
>     >
>     > One question is what this "nonsense" value should be.Ã‚Â  I think it
>     > should be a fixed value. For example 0.0.0.0.Ã‚Â  We would not want a
> problem? Â 

Currently it doesn't. One of my proposals is for each system to generate
a RefID and use that for all interfaces. I'm not sure if that works for
all cases or causes other issues.

Danny

---

From: Sharon Goldberg <[email-masked]>
Date: Tue, 5 Apr 2016 16:54:17 -0400

> all cases or causes other issues.

OK. To clarify: you are proposing an extension to NTP where a server that
has multiple IP addresses can choose a special value as its refID, and
communicate that to its clients.  Then, if the clients choose to take time
from that server, they put that special value in as their refID, instead of
an IP address.

AFAIK this functionality is not part of NTP today, right? It would probably
require an extension field? Also its not backwards compatible, since a
server that chooses to do this would need to make sure that its clients
understand what it is doing. And currently clients don't know how to do
this, right?

Please clarify if I have misunderstood.

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 05 Apr 2016 21:14:20 +0000

Danny Mayer writes:
> > =
> >     > To follow up on my comments on draft-stenn-ntp-suggest-refid-00 at =
> the
> >     > IETF'95 WG meeting just now. The current draft requires the use of =
> an
> >     > extension field.=C3=82=C2  I believe the goals of the draft can be
> >     accomplished
> >     > The goal of the draft is to limit the information exposed by the RE=
> FID
> >     system A. =C3=82
> >     > This proposal allow system A to limit the info it leaks in its refI=
> D,
> >     > without harming any of its legacy clients.=C3=82
> >     >
> >     > Suppose system A is taking time from system B. Then there are two c=
> ases:
> >     > 1) If A gets a time query from system B, A puts the IP of B in the =
> refID
> >     > of its response. This way, even a legacy B can tell it cannot take =
> time
> >     > 2) If A gets a time query from system C, A puts a "nonsense" value =
> in
> >     > its refID.=C3=82=C2  Even a legacy C can see that its IP is not in =
> the
> >     refID,
> >     > and so it is allowed to take time from A.=C3=82
> >     >
> >     > One question is what this "nonsense" value should be.=C3=82=C2  I t=
> hink it
> >     > should be a fixed value. For example 0.0.0.0.=C3=82=C2  We would no=
> t want a
> >     > randomly-chosen value since this might collide with actual IP addre=
> sses
> >     > on the network.
> >     >
> >     > Thanks,
> >     > Sharon
> > =
> 
> >     The problem that I was alluding to in the jabber room is this. I'll k=
> eep
> > =
> > =
> > =
> > problem? =C2 =
> > =

It *hasn't* been an issue.  It would be easy enough to address it if
somebody felt there was a need to, however.

The point is that with your proposal, Sharon, I'm not seeing that it is
possible to address this case.  That is separate from whether or not
it's a problem that needs to be addressed, now or in the "near" future.

It's clear that if A has multiple IPs on it (very common now, not common
20 years' ago) that A could check all of its IPs when it saw B's refid
and decide if any of them meant B was sync'd to A.

In reality, we have only seen that if A sends a packet to B, B responds
to that same address on that same IP.  We have had cases where that
response has come over a different interface.

There are other possiblities, though.

Here are more scenarios.

A VM host maintains the primary clock, and "guests" use that clock.  In
this case, if A and B are on the same VM host, their underlying system
clock is actually the same one.

And that assumes one (common) scenario - the VMs are likely set up so
that all the VMs have tightly-synchronized time.

There's another use case I've been looking for - a VM setup where I
can configure each VM to have a clock that behaves independently, so I
can test wider behaviors more easily.

Is it sufficient to check for loops of degree 1?  Exactly what problems
are we solving here?

What about the situation where a single refclock is directly feeding
multiple servers?  If A and B are getting their time from the same
refclock, should that mean A and B should not fall back on each other if
one of them loses signal to their shared refclock?

If A and B are talking to the same type of refclock is that something
that should be addressed?  There have been cases where broken firmware
has caused a family of refclocks to all take simultaneous leaps to the
same spot in the weeds.

If A and B are both using GPS (for example) and there is a system glitch
where GPS time is affected, is that also a timing loop that we'd want to
avoid?  This happened in Jan 2016.

What about a group of systems that share a PPS signal?  The PPS signal
might become untrustworthy, or the "source of numbered seconds" might
break.

Many of these issues are currently(!) insignificant in a LAN setting
where we're looking at timing/frequency sync to around the millisecond
level (roughly).  I don't *know*, but I'd bet they are significant when
trying to get to the microsecond level, let alone the nanosecond level.
Hello PTP.  Hello RADclock and the work PHK is doing with the algorithms
he's using on Ntimed.  Once we learn more about these, we need to make
sure these different choices will "play nice" with each other.

Similarly, chrony uses algorithms that are slightly different from the
ones NTP uses.  We haven't seen studies of any potential loop
instabilities between these two approaches, either.

I mention this because 20 years' ago we thought we were doing well
getting clocks to sync to within a tenth of a second.  Now, getting
sub-millisecond sync on a LAN is common.  I've spoken with folks who
told me that with some care and support, they are getting sync levels
with NTP that are very close to what they get with PTP - down to the
range of nanoseconds.  Where will we be 10 years' time from now?

I recall PHK saying that with PTP (or PTP-like stuff), he was able to
get offset and frequency sync on cold start in less than a second.  He
was doing this with a poll interval of about -9, or about 500
polls/second.

I know DLM was solving a real and evidenced problem with the refid and
loop detection.

It's looking like we need to take a stronger look at this to decide if
those problems are still significant, and if different problems are
becoming significant.
-- 
Harlan Stenn <[email-masked]>
http://networktimefoundation.org - be a member!

---

From: Harlan Stenn <[email-masked]>
Date: Tue, 05 Apr 2016 21:33:56 +0000

Sharon Goldberg writes:
>>>     The problem that I was alluding to in the jabber room is >>>
>>>     this. I'll keep the problem simple and to one example.
>>>
>>>     A takes its time from B over an IPv4 address. A then gets a time
>>>     query from C over IPv6. How does it know if B and C are the same
>>>     system or different system? A has no information about that.
>>>
>>>     Is this clearer?
>>>
>> Currently it doesn't. One of my proposals is for each system to
>> generate a RefID and use that for all interfaces. I'm not sure if
>> that works for all cases or causes other issues.
> 
> OK. To clarify: you are proposing an extension to NTP where a server
> that has multiple IP addresses can choose a special value as its
> refID, and communicate that to its clients.  Then, if the clients
> choose to take time from that server, they put that special value in
> as their refID, instead of an IP address.

My proposal for SUGGEST-REFID is simply a mechanism for an NTP instance
to tell others "If you use me as your system peer, please use <FOO> as
your refid."

If a server chooses to suggest a single nonce for all of its interfaces
as its suggested refid, great.  If it chooses to use that same nonce for
all of its associations, that's great too.

For me, we're now in the realm of "policy", and there are tradeoffs
there.

My goal was to provide a robust mechanism to allow for these policy
choices.

> AFAIK this functionality is not part of NTP today, right? It would
> probably require an extension field? Also its not backwards
> compatible, since a server that chooses to do this would need to make
> sure that its clients understand what it is doing. And currently
> clients don't know how to do this, right?

Danny's (policy) proposal would most likely be implemented using the
suggest-refid EF proposal I made, so:

Yes, an EF

Nobody is doing this yet, unless somebody has implemented my proposal
and not told me.

As for backward compatibility, that's not really an issue.  An old
server will not understand the EF so it will use the current rules for
choosing its refid when selecting its system peer.

This is what I talked about in my first "set", covering the I-Do and
Last-EF proposals.  If I send you packets with I-Do and get back a plain
response, I know you can't or aren't willing to do these newer choices.
Either way, I need to be prepared to check the legacy and suggested
refid choices in responses I get from you.

If I send you an I-Do and get *no* response, then I don't know if you
never got my packet or you got it and dropped it because it had an
unrecognized EF in it.  Eventually I would just send plain packets.

H

---

From: Danny Mayer <[email-masked]>
Date: Tue, 5 Apr 2016 22:02:14 -0400

On 4/5/2016 4:54 PM, Sharon Goldberg wrote:
>     >Â  Â  Â The problem that I was alluding to in the jabber room is this. I'll keep
>     >Â  Â  Â the problem simple and to one example.
>     >
>     >Â  Â  Â A takes its time from B over an IPv4 address. A then gets a time query
>     >Â  Â  Â from C over IPv6. How does it know if B and C are the same system or
>     >Â  Â  Â different system? A has no information about that.
>     >
>     >Â  Â  Â Is this clearer?
>     >
>     > problem? Ã‚
> that has multiple IP addresses can choose a special value as its refID,
> and communicate that to its clients.Â  Then, if the clients choose to
> take time from that server, they put that special value in as their
> refID, instead of an IP address.
> 
> probably require an extension field? Also its not backwards compatible,
> since a server that chooses to do this would need to make sure that its
> clients understand what it is doing. And currently clients don't know
> how to do this, right?
> 

You did misunderstand. I said the Refid itself and not an extension
field so it would be interoperable. I'm of the opinion that a server
should only have one refid for all interfaces no matter which one is
being used to send the packet. I recall that there is an issue that
Harlan had to explain to me in excruciating detail as to why the
existing refid is the way it is today but I don't recall now the
details. We already have a problem where the hash of the IPv6 address
can conflict with a real IPv4 address.

Danny

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 06 Apr 2016 03:56:24 +0000

> can conflict with a real IPv4 address.

If that would be your policy then the way to implement it is with the
suggest-refid EF proposal I submitted.

The legacy behavior is that "For S2+, I use the IPv4 (or partial hash of
the IPv6) address of my system peer as the refid."

If this situation is not clearly described in my proposal please show
where that information could be improved.

H

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 6 Apr 2016 10:40:46 +0200

> to tell others "If you use me as your system peer, please use <FOO> as

I think an extension field that would allow servers to announce their
own refids to clients is an excellent idea and I'm glad someone is
working on it.

However, I'm not sure about the intended security use case of the
proposal. I'm still not convinced that hiding refid would be a major
problem for an attacker. I think the number of servers that most
clients use on the Internet is fairly small, it probably consist of
pool.ntp.org and the list of public servers maintained on the ntp.org
wiki. The attacker can probably restrict the list by geo location of
the client to a country zone with few hundred of servers at most. I
suspect in most cases the attacker would be able to succeed by sending
spoofed packets from all these servers.

If there is a security issue with clients accepting spoofed packets, I
think it should be fixed directly instead of adding some security
through obscurity.

I'd rather see this extension field used as a way to improve the
robustness of the protocol. As Danny pointed out, if servers announced
the same refid to all clients, they could reliably detect multihomed
hosts.

My suggestion is to modify the field so it includes also refids of all
upstream servers. The clients could then prevent synchronisation loops
longer than 2. Loops between 3 peers polling one another can happen
easily and currently the best recommendation to prevent them is to
use a "line" configuration for the peers instead of a fully connected
mesh. Also, if the clients knew all upstream refids, they could detect
duplicate servers in the paths to the stratum 1 servers and ignore
them or decrease their weight.

-- 
Miroslav Lichvar

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 06 Apr 2016 09:11:19 +0000

> spoofed packets from all these servers.

How do you define "the attacker would be able to succeed" in this case?

Refclocks are increasingly easy to use, too, and this sort of
knowledge/attack is not useful if somebody has a local refclock.

I don't know how valid it is to assume your geo location claim.

Finally, if an attacker is going after somebody's servers that means
the "victim" will be getting unsolicited responses, and we now alert on
that case.

There are other protections against these attacks in the current
codebase, too.

> think it should be fixed directly instead of adding some security

What do you suggest?  How does the current proposal make things worse in
any way?

> robustness of the protocol.

Huh?  I think you are conflating things here.  The proposal is a
*mechanism* and the value it contains is *policy*.

> As Danny pointed out, if servers announced

Then all a bad guy has to do is to send a time request to a target, see
what refid it gets, and then use that knowledge as an attack vector.

Also, there has been no demonstrated benefit that this "wider scope"
loop detection will be of any useful value.

> them or decrease their weight.

Can you demonstrate tangible problems with the current scheme and
tangible benefit with your solution?  Please also consider the security
ramifications.

I've seen lots of mesh setups - they have all worked fine.

H

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 6 Apr 2016 12:39:12 +0200

> How do you define "the attacker would be able to succeed" in this case?

Sending a packet that will be accepted by the client and disrupting
the synchronisation.

> knowledge/attack is not useful if somebody has a local refclock.

Well, if everyone had a reliable refclock, we wouldn't need NTP. :)

> What do you suggest?  How does the current proposal make things worse in

If the implementation follows the recommendation to change the refid
randomly, a client can't detect if it's talking to the same server as
its server for instance.

>From how I read the draft, it seems to suggest it's a security
mechanism. The word "attacker" is used quite a few times. I'd rather
see the proposal going in the other direction, giving clients a better
view of the NTP network instead of hiding information.

If you don't see the benefits, that's ok. I just thought I should
write my suggestions and see if anyone agrees with that.

> what refid it gets, and then use that knowledge as an attack vector.

If the client is fixed to not accept spoofed packets, what exactly the
attacker can do with that information?

> tangible benefit with your solution?  Please also consider the security

For instance, if the client knew the same stratum 1 server is used by
multiple stratum 3 servers and they didn't agree with other sources,
they could be rejected as one assuming the stratum 1 server is bad.

> I've seen lots of mesh setups - they have all worked fine.

I've received reports on loops forming between three peers that
were polling one another when they stopped receiving time from their
upstream sources. I was able to reproduce that. It's not a critical
issue, but I think it would be nice if NTP could prevent that.

-- 
Miroslav Lichvar

---

From: Hal Murray <[email-masked]>
Date: Wed, 06 Apr 2016 13:20:48 -0700

[refid discussion]

[email-masked] said:
> If you don't see the benefits, that's ok. I just thought I should write my
> suggestions and see if anyone agrees with that. 

I think that's an important idea.  We need to consider the trade-offs between 
security from hiding the real refid compared to the forensic value of being 
able to figure out what is or was going on.

On the other hand, I don't have a handful of stories where it has been 
helpful so maybe it isn't really important.  Is a warm-fuzzy feeling worth 
anything?

> attacker can do with that information? 

It tells the bad guys one of the servers a system is using.  That is probably 
useful targeting information, either for DoS or for infiltration via phishing 
or exploit.

[email-masked] said:
> I've received reports on loops forming between three peers that were polling
> one another when they stopped receiving time from their upstream sources. I
> was able to reproduce that. It's not a critical issue, but I think it would
> be nice if NTP could prevent that. 

Doesn't that case bump the stratum level each time around the loop until is 
falls off the end?

-- 
These are my opinions.  I hate spam.

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 06 Apr 2016 21:52:29 +0000

> Sending a packet that will be accepted by the client and disrupting

I think we're approaching a workable definition there.  It depends on
"accepted by the client" and "disrupting".

To me, the goal of the protocol includes (and this needs more tweaking,
too):

- weed out flagrantly bad packets
- update some core peer state data
- perform  thorough packet checks
- if these checks pass, send the packet to the filters

Even then, bogus or otherwise bad packets will get thru.  The algorithms
are designed to deal with these problems.

> Well, if everyone had a reliable refclock, we wouldn't need NTP. :)

I notice the smiley.  Even so, that's an incomplete, myopic, and
insufficient oversimplication.

> randomly, a client can't detect if it's talking to the same server as

I'm not seeing where you think the current proposal is worse in any way.

As for your response/suggestion, where is this recommendation to change
the refid randomly?

In the case where I suggest a refid and you choose me as your server and
use my suggested refid as your system peer, even if I change this on
every poll interval it will be pretty easy for me to track these random
refids and still detect a loop.  Yes, it means saving some state.  The
server suggesting the refid has control over if it uses the same refid
for everybody or not, and also for how often it chooses to change its
suggested refid.  This is a local policy choice.

Even if the client is broken and uses us as their server giving us a
refid we do not understand, in that case there will be a timing loop and
each poll interval the stratum on each side will increase.  Eventually,
these systems will either find another peer to get time from or they'll
both become unsync'd (because there is no other source of time).  This
is no different from A noticing that B is getting its time from A,
refusing to take time from B, and either getting time from somewhere
else or just changing its state to unsynch'd (S16).

> view of the NTP network instead of hiding information.

Yes, it's a security mechanism.  And please remember that we're trying
to get away from the expectation that an S2+ refid is an IPv4 address,
because that is increasingly incorrect.  Also note we're about to make
it easy to control "who can see what", so if you want to know who my
upstream server really is and you have the authorization to get that
information, that would be made available thru a mode 6 query.

> write my suggestions and see if anyone agrees with that.

I wouldn't do what you and Danny are suggesting, but if you want to do
things that way it's your choice - that's why I'm saying my proposal
offers a mechanism, and while we'll provide a default policy choice on
what that suggested refid will be, there will also be mechanism in the
config file to allow you to specify your local policy on what that
suggested refid will be and the circumstances under which that
suggesteed refid will change.

> attacker can do with that information?

How can anybody detect that an incoming packet has a forged source
address?

> they could be rejected as one assuming the stratum 1 server is bad.

How would you propose this work in practice?

This seems to be a special case of a Sybil attack.  I think the overhead
this would introduce would be huge and is better addressed elsewhere, by
proper monitoring and having enough upstream servers.

> issue, but I think it would be nice if NTP could prevent that.

Were these 3 machines the only source of time?  Are they set up in
orphan mode?

What would have happened if this loop had been prevented?

H

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 07 Apr 2016 05:48:36 +0000

Hal Murray writes:
> I think that's an important idea.  We need to consider the trade-offs
> between security from hiding the real refid compared to the forensic
> value of being able to figure out what is or was going on.

As I said before, the real source of time is available from a mode 6
query.

How can y'all not know this, even without my pointing it out?

H

---

From: Hal Murray <[email-masked]>
Date: Wed, 06 Apr 2016 23:21:37 -0700

[email-masked] said:
> As I said before, the real source of time is available from a mode 6 query. 

My system doesn't automatically log that.  The remove system probably does 
log it, but I probably don't have convenient access to those log files.

If the reason the remove server is hiding the real refid is to prevent the 
bad guys from figuring out which servers it is using, then mode 6 access is 
probably blocked.

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 07 Apr 2016 06:35:30 +0000

Hal Murray writes:
> My system doesn't automatically log that.  The remove system probably
> does log it, but I probably don't have convenient access to those log
> files.

How did you get from a mode 6 query to a log file?

> If the reason the remove server is hiding the real refid is to prevent
> the bad guys from figuring out which servers it is using, then mode 6
> access is probably blocked.

a) the "real refid" is not being hidden.  The express purpose of the
refid is maintained.

b) if you have a valid reason for issuing mode 6 queries and the "other
side" doesn't choose to allow this, that's their policy choice.  In that
case you can continue to use them for time, or not.  Your policy choice.

H

---

From: Hal Murray <[email-masked]>
Date: Thu, 07 Apr 2016 01:32:21 -0700

> How did you get from a mode 6 query to a log file?

You said I could get the real refid via mode 6,
  
The context I was thinking about was forensics.  I'm often interested in what 
has happened rather than what is happening.  In that case, all the evidence 
that is available is what's in the log files.

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 07 Apr 2016 08:52:31 +0000

> You said I could get the real refid via mode 6,

No, I said you could get the address of the upstream peer with mode 6.

> The context I was thinking about was forensics.  I'm often interested
> in what has happened rather than what is happening.  In that case, all
> the evidence that is available is what's in the log files.

That may be OK with IPv4 refids, but how do the current logfiles help
you with IPv6-based refids?

H

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 7 Apr 2016 10:57:06 +0200

> able to figure out what is or was going on.

I agree.

> helpful so maybe it isn't really important.  Is a warm-fuzzy feeling worth 

I think it could be more than just a warm-fuzzy feeling. I think
knowing refids up to stratum 1 could be useful for NTP clients to make
better decision in the source selection and also for administrators
when debugging problems.

Harlan referred to mode 6. That's not always available. It's not a
part of the current NTPv4 spec (it was optional in the NTPv3 spec) and
AFAIK only ntpd implements it. Also, on servers accessible from
Internet it's normally disabled as it allows amplification.

> Doesn't that case bump the stratum level each time around the loop until is 

It does. The trouble is that unless the servers have a local reference
enabled (e.g. orphan mode), they will stuck to stratum 16, 15, ... and
will be unusable for clients one, two, ... layers below.

As I said, I don't think it's a huge deal, but it would be nice if NTP
could prevent that.

-- 
Miroslav Lichvar

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 7 Apr 2016 11:32:25 +0200

> As for your response/suggestion, where is this recommendation to change

Your draft says:
  A system that decides to send a Suggested REFID extension field
  SHOULD generate a new Suggested REFID for each new association.

I'm assuming this is about client associations distinguished by IP
address (ignoring the fact the multiple clients may be behind one
address). If the server sends different refids to different clients,
the clients won't be able to detect that they are a client of another
client of the same server.

Older ntpd versions had a check for that. While we may argue about how
useful it really is, it will no longer be possible if the suggestion
is implemented.

> How can anybody detect that an incoming packet has a forged source

The cookie (aka origin timestamp) in the reply doesn't match the request.

> proper monitoring and having enough upstream servers.

It would complicate the selection algorithm, no doubt about that,
but that's the choice of the implementation.

> Were these 3 machines the only source of time?

Yes.

> Are they set up in orphan mode?

No.

> What would have happened if this loop had been prevented?

Alarms monitoring the synchronisation would not be triggered. The
servers would stay in a synchronised state with a real reference and
could be used by their clients and clients of their clients for few
hours until the upstream servers were reachable again.

-- 
Miroslav Lichvar

---

From: Sharon Goldberg <[email-masked]>
Date: Thu, 7 Apr 2016 06:28:27 -0400

My proposal is orthogonal to what is being discussed here.   I believe it
can be implemented alongside Harlan's proposal in suggest-refid.

I want to emphasize that my proposal is backwards compatible and can be
implemented today without an extension field.  Whereas Harlan's idea is
super clever but it requires both the client and the server to understand
an extension field.

First off, to reiterate why I care about this, pretty much all the off-path
attacks that we and Cisco have done have the following first step:

Query the target client. Get the response. Learn its refID. Now you know
the client's server. $Attack the server. (Potentially rinse and repeat if
the client moves to another server.)

Sure, mode 6 reveals this information but modern NTP configurations should
prevent remote IPs from launching mode 6 queries, as it is not only an
amplification vector and also leaks state making it easier to attack the
system.  (See also Cisco's "origin leak" bug [1].)  So I do not have a
"warm fuzzy" feeling when I see NTP leaking information. Especially since
most NTP traffic is not usually cryptographically protected.  I think NTP
should leak less information, not more.

My proposal offers a backwards compatible way for a system to limit the
information leaked in its reference ID. It solves the IPv6 refID collision
problem.  And it seems to be compatible with what NTP is doing today.

Some exceptions could be made for stratum 1 servers to broadcast to all
clients that your refID is GPS or some other external source, if this
really a problem.  But we should be sure its really a problem, because
information leakage always makes it easier to launch attacks.

Much of this discussion has been regarding a future use case in which a
client had two IP addresses B and C.   This client sends a query from
interface B and wants to make sure it is not in a timing loop with another
system A who is taking time from interface B.

One could deal with this usecase using Harlan's proposal, but this does not
preclude my proposal running in parallel.  To deal with this, the client "B
and C"  uses Harlan's extension field to signal "if you take time from me,
put this nonce as your refID."   If it decided to signal the *same* nonce
to all clients, it has explicitly taken on the risk than an attacker can
use this information as a first step in an attack, as I have described
above.  If it signals a different nonce to all clients, then it is
protected from this.

This non-backwards compatible feature would therefore only be used by
systems that need it, eg those that have two IP addresses. All other
systems could limit the amount of information being leaked in their refID
by using my proposal.

Sharon

[1] http://www.talosintel.com/reports/TALOS-2016-0078/

---

From: Danny Mayer <[email-masked]>
Date: Thu, 7 Apr 2016 13:48:26 -0400

On 4/7/2016 6:28 AM, Sharon Goldberg wrote:
> systems could limit the amount of information being leaked in their
> refID by using my proposal.
> 

The number of systems that don't have multiple addresses these days can
probably be counted on one hand. All newer O/S's support both IPv4 and
IPv6 addresses. Furthermore NTP has broadcast and multicast modes so
packets can also arrive over those addresses as well.

Harlan and I had a discussion on this yesterday and now I remember why
the current preferred server's address is being used in the refID. At
the time it was implemented it was meant to prevent timing loops and
security was not a worry. In today's world that's the primary worry.

Danny

---

From: Danny Mayer <[email-masked]>
Date: Thu, 7 Apr 2016 14:31:21 -0400

Danny

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 07 Apr 2016 18:53:02 +0000

> can be implemented alongside Harlan's proposal in suggest-refid.

Agreed.

> the client moves to another server.)

The bad guy can impersonate the client and "vex" the server, and the bad
guy can also impersonate the server and "vex" the client.

> information leakage always makes it easier to launch attacks.

The refid changes affect S2+ only, we'd leave S1 refids alone.

> system A who is taking time from interface B.

The timing loop issue has 2 aspects (at least).  I'll talk about one of
them now.

One reason to avoid a timing loop has to do with sailing off into the
weeds.  I'm not sure how significant this case is anymore.  If I only
have 1 source of time, well, we know what that means.  If that source of
time only has me as their source of time then the timing loop stuff
means we'll detect this immedately and we'll both go out-of-sync.  But
if we do not detect this loop, if A is at S2 then B is at S3.  When
we're down to no additional sources of time, at that point A will become
S4 and B will be come S5, then it goes to S6/S7, ... until the machines
go out of sync.  In this case that will happen in 6 poll intervals.

If there are no other sources of time available, then is it significant
that the machines go out of sync after 1 or 2 poll intervals instead of
6?

I'm using round numbers here.

A semblance of order would be in place if the boxes were configured
using orphan mode, or even local refclocks.

So I'd really like to know exactly what the concerns are about this
particular "failure mode".

H
--
> --001a1142b68e529239052fe28ce7
> Content-Type: text/html; charset=UTF-8
> Content-Transfer-Encoding: quoted-printable
> 
> <div dir=3D"ltr"><div><div><div><div>My proposal is orthogonal to what is b=
> eing discussed here.=C2=A0=C2=A0 I believe it can be implemented alongside =
> Harlan&#39;s proposal in suggest-refid.<br><br>I want to emphasize that my =
> proposal is backwards compatible and can be implemented today without an ex=
> tension field.=C2=A0 Whereas Harlan&#39;s idea is super clever but it requi=
> res both the client and the server to understand an extension field.<br></d=
> iv><br>First off, to reiterate why I care about this, pretty much all the o=
> ff-path attacks that we and Cisco have done have the following first step:<=
> br><br></div>Query the target client. Get the response. Learn its refID. No=
> w you know the client&#39;s server. $Attack the server. (Potentially rinse =
> and repeat if the client moves to another server.)<br><br></div><div>Sure, =
> mode 6 reveals this information but modern NTP configurations should preven=
> t remote IPs from launching mode 6 queries, as it is not only an amplificat=
> ion vector and also leaks state making it easier to attack the system.=C2=
> =A0 (See also Cisco&#39;s &quot;origin leak&quot; bug [1].)=C2=A0 So I do n=
> ot have a &quot;warm fuzzy&quot; feeling when I see NTP leaking information=
> . Especially since most NTP traffic is not usually cryptographically protec=
> ted.=C2=A0 I think NTP should leak less information, not more.<br><br></div=
> ><div>My proposal offers a backwards compatible way for a system to limit t=
> he information leaked in its reference ID. It solves the IPv6 refID collisi=
> on problem.=C2=A0 And it seems to be compatible with what NTP is doing toda=
> y.<br></div><div><div><br></div>Some exceptions could be made for stratum 1=
>  servers=20
> to broadcast to all clients that your refID is GPS or some other=20
> external source, if this really a problem.=C2=A0 But we should be sure its=
> =20
> really a problem, because information leakage always makes it easier to=20
> launch attacks. <br><br></div><div>Much of this discussion has been regardi=
> ng a future use case in which a client had two IP addresses B and C.=C2=A0=
> =C2=A0 This client sends a query from interface B and wants to make sure it
>  is not in a timing loop with another system A who is taking time from=20
> interface B.=C2=A0 <br><br>One could deal with this usecase using Harlan&#3=
> 9;s proposal, but this does not preclude my proposal running in parallel.=
> =C2=A0 To deal with this, the client &quot;B and C&quot;=C2=A0 uses Harlan&=
> #39;s extension field to signal &quot;if you take time from me, put this no=
> nce as your refID.&quot;=C2=A0=C2=A0 If it decided to signal the *same* non=
> ce to all clients, it has explicitly taken on the risk than an attacker can=
>  use this information as a first step in an attack, as I have described abo=
> ve.=C2=A0 If it signals a different nonce to all clients, then it is protec=
> ted from this.</div><br></div>This non-backwards compatible feature would t=
> herefore only be used by systems that need it, eg those that have two IP ad=
> dresses. All other systems could limit the amount of information being leak=
> ed in their refID by using my proposal.<br><div><br><div>Sharon<br><br>[1] =
> <a href=3D"http://www.talosintel.com/reports/TALOS-2016-0078/">http://www.t=
> alosintel.com/reports/TALOS-2016-0078/</a><br></div><br><div><br></div></di=
> v></div>
> 
> --001a1142b68e529239052fe28ce7--
> 
> --===============5243542199967168099==
> Content-Type: text/plain; charset="us-ascii"
> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
> 
> _______________________________________________
> ntpwg mailing list
> [email-masked]
> http://lists.ntp.org/listinfo/ntpwg
> --===============5243542199967168099==--
>