Subject: [ntpwg] Autokey, Shared keys

From: Hal Murray <[email-masked]>
Date: Wed, 22 Jun 2016 23:54:33 -0700

[email-masked] said:
> A legacy MAC has not structure. If autokey is not in use then there are two
> bytes that are known to be zero. Since autokey caches its keys in the key
> cache but does not create an association on the server, the fact that we may
> not know the source doesn't change whether or not it might be autokey, so if
> autokey is in use between our system and any other system, then a legacy MAC
> might contain any bit pattern. ...

Is anybody using autokey?

Even if there are a few users in obscure corners, do we want to spend time 
trying to make new header parsing rules work with autokey?

My vote is drop it and stop wasting time discussing anything related to 
autokey.

How often are shared keys used?  I had never used one until recently when I 
setup a test case.  I don't remember much discussion, but NIST does offer it.

Shared keys already require per key manual setup for each key, so a bit more 
work doesn't seem unreasonable.  I expect any sane setup would use a key on 
only one host.  (Maybe one company/organization but they probably use a 
single CIDR block.)

Could we switch to a clean new EF format by requiring new servers that want 
to support old style shared keys to specify the IP ranges for systems that 
use old style extensions, aka shared keys?  It would be a simple edit to the 
keys file.

-- 
These are my opinions.  I hate spam.

_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 23 Jun 2016 12:11:10 +0000

> Is anybody using autokey?

That's a great question.

Anyone?

---

From: Brian Utterback <[email-masked]>
Date: Thu, 23 Jun 2016 09:43:31 -0400

> Is anybody using autokey?

Probably. We do see customers call in with questions about autokey setup 
and usage. I have no way to know if it is idle curiosity and they all 
have abandoned it or if it is an integral part of they NTP network. But 
we have enough that my frontline engineers have gotten pretty good at 
setting it up.

> trying to make new header parsing rules work with autokey?

Autokey already conforms to all the proposed extension field parsing 
rules. The problem is the legacy MAC field.
> My vote is drop it and stop wasting time discussing anything related to

I would humbly suggest that the minimum prerequisite for dropping 
autokey support entirely is a working replacement for the feature.

> setup a test case.  I don't remember much discussion, but NIST does offer it.

All the time. No shared keys, not administrative control. No shared 
keys, no secure peering. And unlike autokey, shared keys are supported 
by multiple implementations. No shared keys and we lose a bug chunk of 
backwards interoperability.

> use old style extensions, aka shared keys?  It would be a simple edit to the

Thus requiring even more information from outside the packet. This is 
not a hard problem, it is almost trivial to solve. We just need to agree 
how to do it and agree to do it and then document it so it doesn't break 
in the future.

There are three rules currently in place that make this work:

1. A legacy MAC, if present, is the last thing in the packet.
2. A legacy MAC is 24 bytes or less.
3. The last extension field in the packet, if any, must be more than 24 
bytes.

All we have to do is agree to keep these rules, document them as "MUST"s 
in the RFCs to avoid future breakage, and specify what do do if you 
don't have any extension fields to add that are more than 24 bytes. It 
is completely backwards compatible, it is current practice and it 
doesn't add any new restrictions that are not already in place 
implicitly. Notice autokey is not an issue here, it is not relevant. And 
it allows parsing of the packet without any outside knowledge. The only 
thing new that needs to be added is how to guarantee rule 3.

If anyone wants to propose changing any of these requirements then they 
must say how parsing will be done with their proposed change.

Now, on the assumption we are willing to agree on the rules above, then 
how do we guarantee rule 3? Current proposals are:

1. Pad the last EF.
2. Have a no-op EF that is more than 24 bytes.
3. Have a "packing" EF that allows you to pack multiple EFs into one 
larger one.
4. Have a "LAST" EF to make it easy to know when the last EF has been 
reached.

Any others?

Comments:

1. Eliminates the ability of an EF to use length to convey information.
2. Seems wasteful, the standard NTP packet is only 48 bytes to begin 
with. More wasteful than item 1 since at least with item 1 the waste is 
(24 bytes - original size of EF) instead of a guaranteed 24 bytes.
3. Make parsing somewhat more complex, and doesn't eliminate the problem 
since all of your EFs together may not by 24 bytes. Could be combined 
with item 1 to make the necessary padding potentially even less.
4. This item only works if it is a "MUST" and thus fails backwards 
compatibility. We would have to special case existing formats by 
detecting the autokey extension fields which would tell us that a legacy 
MAC must be present but that a LAST may not be present, which gets us 
back to the same problem, unless we add more restrictions such as that 
an autokey EF "MUST NOT" be used with any other extension. Then we have 
to deal with the problem of how deal with someone who wants to use the 
nifty new EFs *and* autokey. Just saying "can't" really isn't acceptable 
until there is a working replacement for autokey.
-- 
Oracle <http://www.oracle.com>
Brian Utterback | Principal Software Engineer
Phone: +1 6038973049 <tel:+1%206038973049>
Oracle Systems/RPE Solaris Network
1 Oracle Dr. | Nashua, NH 03062
------------------------------------------------------------------------
All working systems eventually show their own agendas.
------------------------------------------------------------------------
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to 
developing practices and products that help protect the environment
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 23 Jun 2016 16:46:58 +0200

On Thu, Jun 23, 2016 at 09:43:31AM -0400, Brian Utterback wrote:
> Now, on the assumption we are willing to agree on the rules above, then how
> do we guarantee rule 3? Current proposals are:
> 
> 1. Pad the last EF.
> 3. Have a "packing" EF that allows you to pack multiple EFs into one larger
> one.
> 2. Seems wasteful, the standard NTP packet is only 48 bytes to begin with.
> More wasteful than item 1 since at least with item 1 the waste is (24 bytes
> - original size of EF) instead of a guaranteed 24 bytes.
> since all of your EFs together may not by 24 bytes. Could be combined with
> item 1 to make the necessary padding potentially even less.

3. can also be combined with 2 if the field length should convey
information (and I agree with you that it would be useful if it did).
The idea was to pack also a no-op or "padding" EF if the real EFs are
not long enough to fill the space in the packing EF.

For example, if an NTP packet should deliver a single 8-octet I-Do EF,
the packet would include also a 16-octet padding EF to fill the
28-octet packing EF. The structure of the packet would look like this:

- NTP packet (76 octets)
  - NTP header (48 octets)
  - Packing EF (28 octets)
    - EF header (4 octets)
    - EF data (24 octets)
      - I-Do EF (8 octets)
        - EF header (4 octets)
        - EF data (4 octets)
      - Padding EF (16 octets)
        - EF header (4 octets)
        - EF data (12 octets)

-- 
Miroslav Lichvar

---

From: Richard Welty <[email-masked]>
Date: Thu, 23 Jun 2016 11:43:46 -0400

> we have enough that my frontline engineers have gotten pretty good at

...

> autokey support entirely is a working replacement for the feature.

it is my understanding that there are people using autokey, and they
would like to get out of it as soon as an alternative becomes available.

the current plan is that autokey will remain in the reference
implementation until NTS is ready. at least, that was the plan the
last time we discussed it.

richard

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 23 Jun 2016 17:13:11 +0000

> > Is anybody using autokey?

Yes.  A disturbing number of people are quietly using it.

H

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 23 Jun 2016 17:48:48 +0000

Brian Utterback writes:
>> Could we switch to a clean new EF format by requiring new servers
>> that want to support old style shared keys to specify the IP ranges
>> for systems that use old style extensions, aka shared keys?  It would
>> be a simple edit to the keys file.

I'm gonna wave my hands again and say there's no real problem with
legacy MAC use, even with the proposed new EFs.  A legacy MAC MAY follow
a LAST-EF packet.

Any MAC in an NTP field "protects" that data that comes before it.

This means that it is POSSIBLE to have multiple MACs in a packet.

If we have: BASE + MAC-EF + LEGACY-MAC

then the base packet is "protected" by both the MAC-EF and the
LEGACY-MAC, and the BASE + MAC-EF is "protected" by the LEGACY-MAC.

> 1. A legacy MAC, if present, is the last thing in the packet.

Agreed.  We SHOULD NOT allow, for example, a subsequent EF-TC in this
situation.

> 2. A legacy MAC is 24 bytes or less.

OK.

> 3. The last extension field in the packet, if any, must be more than 24 

Why is this a necessary restriction?  EFs are "well formed".  Here's the
general algorithm you've been asking for:

When parsing an NTP packet, we produce a list of potential ways the
packet might be parsed.

That list will contain either 0, 1, or more possible ways the packet can
be parsed.

If the packet can be parsed 0 or 1 way, we know what to do.

If the packet can be parsed more than one way, we try all of the
parsings, including attempts to validate any authentication, and we're
left with 0, 1, or more successfuly validated packet interpretations.

If the packet can be interpreted and validated 0 or 1 way, we know what
to do.

If the packet can be interpreted and validated more than 1 way, then we
log this event, and invite the admin to send us this packet and the
supporting information, and we send the admin a t-shirt.  We discuss
this situation with the implementor, and see if they're worthy of a
nomination to a "wall of shame".

I'm still not seeing how this can happen, since:

 it can only happen if a legacy MAC happens to parse as both a legacy
 MAC *and* some combination of new EFs that 1) all contain the
 MAC-OPTIONAL flag and 2) have proper length fields, with no leftover
 data.

Old code will never see this, as old code only knows about legacy MAC.

New code should never see this, as the only way this *could* happen is
if we had:

 BASE + (EF/MAC-OPTIONAL)+ + LAST-EF/MAC_OPTIONAL

*and* the EF data also:

- matched a valid keyID
- provided a crypto match to that valid keyID

In this case we'd have a t-shirt winner.

Note that as written, the LAST-EF proposal is expected to usually be a
4-byte field, with a Field Type of 0x2008 and a Field Length of 0x0004.

If there is a valid concern that there *might* be a problem, we can
certainly allow for a longer LAST-EF packet where that data is simply
padding bytes.  But I'm not seeing the need.

Remember that a legacy MAC is usually going to be 4 bytes of a key ID
followed by 16 bytes of MAC, and for this to be valid we MUST have a
matching key ID in our stored data and the MAC MUST validate against
that key, and we MAY have information that says "the sender is expected
to use this keyID".

> thing new that needs to be added is how to guarantee rule 3.

As I think I have shown above, rule 3 is not needed.

> must say how parsing will be done with their proposed change.

How did I do in this regard?

H
--

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 23 Jun 2016 18:01:34 +0000

> 28-octet packing EF. The structure of the packet would look like this:

No need for this.  The I-DO proposal allows for padding already, so:

 BASE + I-DO/MAC-OPTIONAL

- Base packet
- I-DO/MAC-OPTIONAL
- - Field Type: 0x2007
- - Field Length: whatever
- - (whatever - 4) octets of I-DO data, including any 0x0000 padding

One COULD instead send:

 BASE + I-DO + LEGACY-MAC
 BASE + I-DO/MAC-OPTIONAL + MAC-EF/MAC-OPTIONAL
 BASE + I-DO + MAC-EF + LAST-EF/MAC-OPTIONAL
 BASE + I-DO + MAC-EF + LEGACY-MAC
 BASE + I-DO + MAC-EF + LAST-EF + LEGACY-MAC

In 4 of the 5 items above, I-DO could be I-DO or I-DO/MAC-OPTIONAL.

In the last item above, LAST-EF could be LAST-EF or LAST-EF/MAC-OPTIONAL.

H

---

From: Hal Murray <[email-masked]>
Date: Thu, 23 Jun 2016 11:26:01 -0700

[email-masked] said:
> I'm gonna wave my hands again and say there's no real problem with legacy
> MAC use, even with the proposed new EFs.  A legacy MAC MAY follow a LAST-EF
> packet. 

I'm missing an important idea.  Why is it interesting to put a legacy MAC 
after any new EF headers?

It seems like an unnecessary complication that will be a pain in the ass for 
a long time.

If a packet wants a MAC, why not put it in a new EF?

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 23 Jun 2016 20:26:01 +0000

> It seems like an unnecessary complication that will be a pain in the ass for 

We're going to have to live with legacy MAC for a very long time.

> If a packet wants a MAC, why not put it in a new EF?

One SHOULD do that.  I'm not sure that one MUST do that.

If we have to handle new EFs and a legacy MAC for even 1 case, then it
MUST work for that case.  If it MUST work for that one case, then
there's no clear reason it would not worlk for other cases.

H

---

From: Brian Utterback <[email-masked]>
Date: Thu, 23 Jun 2016 16:42:36 -0400

> Brian Utterback writes:

Let me say that I agree with everything you wrote above this line. Two 
thirds done! Yah!

> How did I do in this regard?

Terrible. 8-(

Ideally, you should not need to examine a piece of data to know what it 
is. We know, for instance that in a NTP packet the byte at offset 1 is 
the stratum. I can read this byte and know what its semantics and valid 
values are with no further information.

Next best would be to read some data at a known location that tells me 
what a particular byte is. For instance, all our problems would go away 
if there were two bytes or even one byte available that could be used as 
a "length of included MAC field". I could then tell you without any 
further info whether or not there are EFs, and if there are, the exact 
offset of the Field Type and Length fields of the first one. And from 
that info, I can tell you without reading it if there is another EF and 
where its Field Type and Length fields are and so on.

The next to last in desirability would be to guess all possible 
allowable structures of possibly several and then look at the data 
involved and figure out which ones give his data that is most likely to 
be valid, knowing that it is at least theoretically possible that there 
is no definitive solution although in practice that won't happen.

The absolute worst is to do the same as in the last paragraph, but to 
require possibly computationally expensive operations to validate some 
of the fields to determine which format is correct, when some of those 
operations require context that is outside of the packet.

The algorithm you described is basically in the worst class that might 
be made to work, while at the same time you hand waved over the tricky 
parts:

"When parsing an NTP packet, we produce a list of potential ways the
packet might be parsed."

At best, if it does work we are trading a trivial solution for a highly 
complex solution for basically no gain.

Worse still, without documenting the algorithm, I doubt that anyone else 
would ever be able to implement it correctly.

I can't find the email right now (if it was an email) but about 10 years 
ago I had an exchange (or read someone else's exchange, or read it in a 
doc or book, whatever) with Dave about why he chose to the length 
restrictions he did for AUTOKEY extensions and the MAC, and he said 
something to the effect that it was long standing best practice 
(tradition?) on the Internet that you should not have to read data to 
know what it is. I would have to concur.

In any case your algorithm doesn't say anything as to why it would be a 
good idea to use it. Show of hands, does anyone else think that the 
complexity of this algorithm is worth it so that we can eliminate the 
need for the minimum EF size restriction, or some other simple 
deterministic solution?
-- 
Oracle <http://www.oracle.com>
Brian Utterback | Principal Software Engineer
Phone: +1 6038973049 <tel:+1%206038973049>
Oracle Systems/RPE Solaris Network
1 Oracle Dr. | Nashua, NH 03062
------------------------------------------------------------------------
All working systems eventually show their own agendas.
------------------------------------------------------------------------
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to 
developing practices and products that help protect the environment
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Harlan Stenn <[email-masked]>
Date: Fri, 24 Jun 2016 04:49:07 +0000

> Let me say that I agree with everything you wrote above this line. Two 

:)

> >   BASE + (EF/MAC-OPTIONAL)+ + LAST-EF/MAC-OPTIONAL
> >
> > *and* the EF data also:
> >
> > - matched a valid keyID

But after the base packet that's not really possible.  The best we can
do after base packet is know that it is "some number of EF" possibly
followed by a legacy MAC.

We have to look at the EF data to see how long it is.  That's "examining
the data".

> operations require context that is outside of the packet.

"possibly computationally expensive operations"?

How often will one need to do the above dance?  It's the fallback case
if the normal stuff doesn't work.

> would ever be able to implement it correctly.

Really?  How hard is it to detect an EF?

4 bytes - a Field Type and a Field Length.  That's the extent of that
parsing.  If the packet is short, this CANNOT be an EF.  If there are 0
or more bytes, then what follows MUST be either zero or more EFs,
followed by zero or one legacy MAC field.  This is easy stuff.

If the EF is unrecognized, we can ignore it.  This begs the question
"Why didn't they use I-DO"?

When we've exhausted the EFs or hit a LAST-EF, what follows MUST be a
legacy MAC or the packet is invalid.

While we're seeing EFs, we maintain two flags, 'mac_needed' and
'mac_seen'.  These are initialized to 0.

Each time we see a new EF we update these:

- if the EF Field Type indicates MAC-REQUIRED, we:

   mac_needed = 1;

- if the EF Field Type indicates MAC-PROVIDED, we:

   mac_needed = 0;
   mac_seen = 1;

If this was a LAST-EF, we're done processing EFs.  Anything after this
is "something else", and today that means LEGACY-MAC.

If you don't want to implement the full algorithm I mention above then
this should be sufficient.

It will not detect the case where we do not provide
LAST-EF/MAC-OPTIONAL, and the packet contains what I wrote above:

:  BASE + (EF/MAC-OPTIONAL)+ + LAST-EF/MAC-OPTIONAL
:
:*and* the EF data also:
:
:- matched a valid keyID
:- provided a crypto match to that valid keyID

Note well this last bit - for this to happen the sender MUST NOT send
LAST-EF, but the legacy MAC MUST match a 24-byte series:

 (EF/MAC-OPTIONAL)+ + LAST-EF/MAC-OPTIONAL

The answer here is obvious:  Don't do that.

If you are going to provide a legacy mac in the presence of other new
EFs, then it's best to send a LAST-EF, *regardless* of the length of the
last EF, because it might be possible that the legacy MAC decodes to:

 (EF/MAC-OPTIONAL)+ + LAST-EF/MAC-OPTIONAL

Please note that the above could be true REGARDLESS of how long the
legacy MAC happens to be.  It could be shorter or longer than 24 bytes.

Also note the odds of this happening is on the order of 1 in 2^190 or so
(a few in 2^192, for a 24 byte MAC).

So given this, I also don't understand why folks think 24 is a magic number
for protecting against this case in the face of a legacy MAC.

> know what it is. I would have to concur.

There are places this is true and places it is not.

At some point one needs to look at the data in order to do something
with it.

> need for the minimum EF size restriction, or some other simple 

The (thorough) algorithm I describe above is a way to robustly process
packets in the vase of a sender that does not send LAST-EF.

To me, it's pretty clear - if a legacy MAC is provided the sender should
also provide LAST-EF if there are any EFs in the packet.

Note that with new code, there is still this ~1 in 2^190 chance that
things will go wrong with a packet from an older client.  In that
instance, what the client sends is:

 BASE + LEGACY-MAC

but we could successfuly parse that as either:

 BASE + LEGACY-MAC

or

 BASE + EF/MAC-OPTIONAL + LAST-EF/MAC-OPTIONAL

What should we do in that case?

I'm pretty sure it doesn't much matter.

H

---

From: "Ulrich Windl" <[email-masked]>
Date: Fri, 24 Jun 2016 09:51:33 +0200

>>> Hal Murray <[email-masked]> schrieb am 23.06.2016 um 08:54 in

> My vote is drop it and stop wasting time discussing anything related to 

But only for NTPv5 AND if it's clearly stated that it won't be able to do any autokey protocol AND if you ensure that any client trying autokey will not confuse an NTPv5 server, AND no NTPv5 reply will confuse any NTPv4 (autokey) client.

> setup a test case.  I don't remember much discussion, but NIST does offer 

We used shared keys ever since wie used NTP (v3). Staring with DES keys, then MD5 keys...

> use old style extensions, aka shared keys?  It would be a simple edit to the 

The problem was that whenever you start using a data format that was not designed to be extensible, you run into problems like these when you actually extended the format in multiple ways... (MHO)

We can't undo, we just can try to do better next time.

Ulrich

---

From: "Ulrich Windl" <[email-masked]>
Date: Fri, 24 Jun 2016 09:55:55 +0200

>>> Brian Utterback <[email-masked]> schrieb am 23.06.2016 um 15:43 in
> we have enough that my frontline engineers have gotten pretty good at 

I guess as long as the NTP project page has no negative statement about autokey, people will find, read and try it.

---

From: Brian Utterback <[email-masked]>
Date: Sat, 25 Jun 2016 14:27:04 -0400

> We have to look at the EF data to see how long it is.  That's "examining

Indeed. I did say "ideally". That isn't possible in our current situation.
>> where its Field Type and Length fields are and so on.

This is the next most desirable situation and it still is possible. Just 
because we can't get the best, that isn't a reason to use the worst.

> if the normal stuff doesn't work.
I don't know what you mean by "the normal stuff", there is no normal 
stuff. And why require it in any case if you don't have to?
> followed by zero or one legacy MAC field.  This is easy stuff.

No it isn't. Without the length restrictions there is no way in the 
first four bytes to distinguish an EF from MAC.

> "Why didn't they use I-DO"?
Because we don't require it. Are you saying you think that it should be 
a requirement that every packet starts with an I-DO field? Or that no 
system ever send any extension field except I-DO until they know that 
the other system understands it?

> legacy MAC or the packet is invalid.

The LAST extension is one of the proposals on the table. Are you 
advocating it? We can't require it since the current autokey code 
doesn't use it. To get it to work we would have to do something like 
require it except when there is autokey and then say the MAC field after 
autokey must have the length restriction and forbid other extensions 
with mixed with autokey. If you do not require the LAST extension then 
it hasn't bought you anything.

> for protecting against this case in the face of a legacy MAC.

It is magical because at the current point in time it is the restriction 
that allows easy parsing and is already in existence.

I think your algorithm makes a few assumptions and forces policy 
decisions. Your full algorithm will probably take a couple of hundred of 
lines as compared to maybe 10 lines. And it is not at all obvious even 
if it works, so if you do not document it no one else is ever going to 
be able to deduce the required algorithm.

> At some point one needs to look at the data in order to do something

It is not forbidden to look at the data. However, it is far preferable 
to look at the data knowing what it is expected to be rather than to 
look at the data and trying to deduce its semantics based on what you 
find. That is the difference between a C language "goto" statement and 
the Intercal "comefrom". In principle they do the same thing, but no one 
would really use "comefrom" because of the increased difficulty of 
writing a correct program and then maintaining it.

> I'm pretty sure it doesn't much matter.

Okay, consider this. I want a policy of ignoring unknown EFs but 
otherwise processing the packet normally. Of discarding immediately all 
non-well-formed packets. And if there is an invalid MAC, to respond with 
the normal NTP packet plus any EF responses that did not require a MAC. 
The packet has 32 bytes beyond the end of the fixed portion. I look at 
the first four bytes and it is an extension I recognize and it is eight 
bytes long and does not require a MAC although one is allowed. I now 
have 24 remaining bytes. I look at the next four bytes. The length 
checks out, but if it is an extension field it is not one I know. What 
do I do now?

---

From: Harlan Stenn <[email-masked]>
Date: Sun, 26 Jun 2016 00:36:27 +0000

> first four bytes to distinguish an EF from MAC.

How is not not easy?  The length restriction really doesn't buy us
anything.

A legacy MAC could be other than 24 bytes long.

We have information we can use if we are considering that the current
data is a potential EF.

We have information we can use if we are considering that the current
data is a potential MAC.

More about this below.

The bottom line is that there are potential problems with the legacy MAC
and there is no simple way to always make the correct decision,
especially in the face of tampering,

> the other system understands it?

Neither.  See below.

> advocating it?

Well, I proposed it and I think it will be very useful in appropriate
circumstances.

> We can't require it since the current autokey code doesn't use it.

We MAY use it, as one would expect an unrecognized EF to be ignored.
But in the same way one should not be using Autokey, one should not be
offering extension fields until things have changed enough so that there
is a reasonable expectation they'll be ignored.

> To get it to work we would have to do something like 

I think it's a bad idea to REQUIRE LAST-EF before any legacy MAC.

Short summary: Issues around autokey and legacy MAC don't seem that
important given how we expect things to go in the future, and all I plan
to do is make sure that whatever active steps folks decide to take to
address these things do not interfere with forward progress.

Here's a hint of what I'll probably finish below.

If all we send is a base packet and a legacy MAC, we still cannot be
CERTAIN that a 24-byte legacy MAC is not a ~1 in 2^190 match to 1-6
potential EFs.

A legacy MAC may be shorter or longer.  The odds in these cases adjust
accordingly.

I'm still firmly in the camp that says "this doesn't matter", even if we
*only* consider legacy 24-byte MACs.

NIST sees 16 billion packets a day.  If EVERY packet they saw had a
legacy mac, there is a random chance they'd see *1* of these collisions
sometime in the next 10^49 days, or once in the next 48 years.

How do different versions of various implementations respond to an
unkonwn EF (LAST-EF)?

Do you really think it will be a good idea to REQUIRE a LAST-EF before a
legacy MAC?  What sort of compliance do you expect we'd see?

> that allows easy parsing and is already in existence.

It may be other than 24 bytes.  It's still an ambiguous situation.

> be able to deduce the required algorithm.

You asked for an algorithm.  I provided it.  I'm not saying we should
use it, even if it addresses this problem.

If we choose to go forward with it it will be documented.

If using the algorithm provides enough benefit to justify its
complexity, great.  If not, we don't use it.

As I mentioned, I think we're putting a HUGE amount of effort into
solving a problem that will likely NEVER happen.

See below.

> writing a correct program and then maintaining it.

Sure.  We're back to "how much effort is it worth taking to deal with
this ambiguity?"

> otherwise processing the packet normally. Of discarding immediately
> all non-well-formed packets. And if there is an invalid MAC, to
> respond with the normal NTP packet plus any EF responses that did not
> require a MAC.

I think your last point is ... wrong and dangerous.

The MAC protects the ENTIRE packet.  If the MAC is invalid (legacy or
otherwise), the packet should not be "considered".  Note that some
timestamps get updated even if the MAC is invalid.

> checks out, but if it is an extension field it is not one I know. What 

Write in for your t-shirt - you found a 1 in 2^190 packet.

If you are using the algorithm above, then you have a list of 2 ways
this packet can be processed.

You know it can be handled as EFs.

See if can be validated as a legacy MAC.

Is there an expectation that this association is using authentication?

If so, then there's your answer.

If not, there is only 1 way to successfully use this packet - as if
it was EFs.

Here's the bit I mentioned above:

Since there is no way to tell if even a base packet + 24 bytes is a
legacy mac or not, I propose the following:

New code will ALWAYS try and parse EFs, up to either:

- where the remaining data does not parse as an EF, or
- up to a LAST-EF

At that point, if there is any data remaining, process it as if it is a
legacy MAC.

In this 1 in 2^190 case, the packet would be parsed as EFs, and would NOT
be parsed for a legacy MAC.  Therefore, if this association was one that
required authentication the packet would be rejected.

New code will negotiate and if there was a cause to use a legacy MAC,
LAST-EF shd be used before it.

Folks who only do no uprades and run only old software will get what
they have always gotten.

When dealing with a mix of old/new software, there's a TINY chance that
the new software will throw out a packet that the old software would
have just treated as a MAC.

When dealing with new software, there's no problem.

I'm fine with this solution.  Or we can go with the algorithm I
described above and we can catch the ~1 in 2^190 case.

Am I missing something?

H

---

From: brian utterback <[email-masked]>
Date: Sun, 26 Jun 2016 00:17:49 -0400

> A legacy MAC could be other than 24 bytes long.

Only if you don't have the length restriction. I don't understand how
you could make those last two sentences together. It essentially boils
down to "a length restriction doesn't buy us anything if we don't have a
length restriction".

> We have information we can use if we are considering that the current

Do you really not understand what a horrible idea it is to have a
non-deterministic predictive algorithm when we could just as easily have
a simple deterministic one?

> especially in the face of tampering,

We have four different proposals on the table, any one of which provides
a simple way to always make the correct decision, but you insist on not
using any of them.

> Well, I proposed it and I think it will be very useful in appropriate

We are not discussing the usefulness of the LAST extension per se.
However, the only think it provides is a way to solve the parsing
problem we are discussing. At some point in time you must have thought a
simple deterministic algorithm was a better solution.

> is a reasonable expectation they'll be ignored.

What needs to change? The RFC says that  now, the current code
implements it.

> address these things do not interfere with forward progress.

If you don't require it, it is useless. If you don't require it if
autokey is in use but do require it in all other cases then it is less
useful than it might have been, but it might still be made to work. If
you never require it then it serves no function at all.

I have been trying since April to implement generalized extension
processing in the stable branch software base. I spent all of my free
time for two weeks trying to do so in the hope that a concerted effort
would do it. Any one of the other proposals would probably take maybe a
day to implement, the only impediment is that we have to pick one and
stick to it and then fix all of the drafts and RFCs to be consistent
with it.

You say that you don't want to make any restrictions that are not
required. You say that we should not be dictating policy. But rather
than just choosing one of the proposed schemes, you want to use a scheme
that depends on future implementers following the same thinking that you
have, with the same future trajectory of use that you envision, all
without even documenting what that is. You had multiple volunteers
willing to contribute their time, thoughts and code to this project and
none of them are currently able to progress at all. I submit that the
you are the one that is interfering with forward progress.

> timestamps get updated even if the MAC is invalid.

So you *do* want to dictate policy? I will point out however, that the
policy that you consider dangerous is the policy that is currently
implemented in the reference implementation.

> Write in for your t-shirt - you found a 1 in 2^190 packet.

Sorry, that is a one in 2^16 chance of happening. Actually, since values
less than the necessary value will present the same problem and just
postpone the issue, it is really 5 in 2^16 at this stage.

> At that point, if there is any data remaining, process it as if it is a

Except that at any point in the chain up to that point could be the
start of the MAC.  If the remaining data doesn't validate, it may be the
case that the previous pointer was part of the MAC and just had a value
that happened to be smaller than the remaining data. So, you would need
to backtrack and try that one. And if that doesn't validate, you'll need
to backtrack again, etc.

> LAST-EF shd be used before it.

You are assuming a MUST without documenting it.

> described above and we can catch the ~1 in 2^190 case.

Yes, the odds are far, far greater than 1 in 2^190, unless you try
actually decrypting. If you do insist on requiring data outside the
packet to parse it you are almost certainly going to piss off your users
someday when some piece of network infrastructure depends on that
capability and it doesn't work with NTP. They are going to say, "it
works with telnet, http, ssh, SMTP, and every other protocol except NTP.
Fix NTP!" Except at that point it will be too late. And you still end up
assuming restrictions and dictating policy with your scheme anyway.

Really, I just don't get it. You have several simple solutions that
definitely work and one complex solution that may or may not work. Even
if you are convinced that it will work, why is a complex solution better
than a simple one? Why use one that goes against multiple long standing
best practices of protocol design? Even if I thought it would work (and
I don't at this point) you have not given any hint of a reason why it
would be superior to the simpler choices.

I have had others convey to me how frustrated they are with this
situation. As near as I can tell, there is literally no one else that
agrees with you on this issue. But you are Zuul, the gatekeeper, you
have the keys. If you choose one of the simple solutions I am prepared
to contribute the code to implement the framework along with changing
the RFCs to be consistent. I would even make a stab at converting the
existing autokey code to use the new framework. However I know I can't
implement your algorithm from your description. If you want to use it,
fine, but I think that you will have to implement it yourself. I
certainly can't and I am pretty sure the others on this list can't
either. When you have working code available, I will be happy to
re-enage and continue from there but until then, I am afraid I have
little I can contribute.

All working systems eventually start to exhibit their own agenda

---

From: Harlan Stenn <[email-masked]>
Date: Sun, 26 Jun 2016 06:55:41 +0000

> down to "a length restriction doesn't buy us anything if we don't have a

The length of a legacy MAC is 24 bytes.  Unless it's 20.  Unless
somebody did something else and some other value is sent.

We should be able to agree that the 2 lengths we're most likely to see
are 20 and 24 bytes.  But other lengths are *possible*.

Perhaps I'm missing the definition of "length restriction".

But folks are still talking about EFs + legacy mac, and how it's
"critical" that the EF before the legacy MAC is at least 28 bytes long.

It complicates things and I don't see that it really provides any
benefit.  It wastes space and consumes network bandwidth needlessly.

> a simple deterministic one?

Are you talking about the more complicated algorithm?  That one is not
non-deterministic.  It's not as simple as the algorithm I propose
actually using, but it's still deterministic and pretty simple.

> a simple way to always make the correct decision, but you insist on not

I'm a bit lost.  I propose we:

- parse/use valid EFs up to either:
- - the end of a LAST-EF
- - what comes next is not an EF
- then treat any remaining data as a legacy MAC

> simple deterministic algorithm was a better solution.

I don't see legacy MACs being used, except when talking with older
clients.

However, one MAY choose to use them anyway, in which case it's perfectly
OK to use:

 BASE + optional stuff + MAC-EF + LAST-EF/MAC-OPTIONAL + LEGACY-MAC

But I think we're rehashing things.  If you insist there's a reason for
this, that's great.  As long as what you/y'all decide doesn't require
excessive restrictions, I don't care.

Saying that any EF before a legacy MAC MUST be 28 bytes or longer is
excessive.

There is no functional difference between trying to parse:

 BASE + LEGACY-MAC

and

 BASE + (EFs that are MAC-OPTIONAL with a length of 24 bytes)

except folks want a rule that says "any EF before a legacy MAC must be
at least 28 bytes long", and in this case that certainly does no good at
all, since parsing the latter option comes across as "EFs with no legacy
MAC".

If this last case is correct then there no difference between it and the
case where there *are* EFs before the legacy MAC.

So it still looks to me like you are fighting for rules that are really
not doing any good.

> What needs to change? The RFC says that  now, the current code

I don't know how much testing has been done regarding unrecognized EFs.

And the rules (before the recent mess) was that the presence of ANY EF
meant a MAC MUST be provided.  That MAC is a legacy MAC.

> you never require it then it serves no function at all.

Then you want to break backward compatibility.

I'm opposed to that.  Strongly.

As I've said before, we have responsibilities *to* people, not *for*
them.

I'm fine with SHOULD.  I'm hesitant about MUST.

> stick to it and then fix all of the drafts and RFCs to be consistent

First, if this was easy we would have found a solution that more folks
like, sooner.  I would have preferred that, but reality is biting.

Are you saying the simple algorithm I have suggested is more difficult
to implement?

I would be surprised if even the more complicated one would take
significantly more time to implement.  It would certainly take more time
to test, and as I've already said, I don't think it's worth the trouble
to do this just for the sake of detecting a 1 in 2^190 collision.

> you are the one that is interfering with forward progress.

I've stated the proposal I think we should use:

- parse/use valid EFs up to either:
- - the end of a LAST-EF
- - what comes next is not an EF
- then treat any remaining data as a legacy MAC

This mechanism needs no additional limitations on MAC length or length
of EF before a MAC or anything else.

To me, it seems the cleanest and simplest solution.

I haven't yet seen what you think of that idea.

> implemented in the reference implementation.

Please show me where.  I see receive() in ntp_proto.c which sets up
is_authentic, which gets passed to AUTH() checks, which determine
responses.  The AF_XMIT case processes no/good auth, sends crypto-nak on
bad auth, and otherwise does not respond.

Similar checks are made for manycast, broadcast client, new passive,
autokey, etc. .

So what am I missing?

> postpone the issue, it is really 5 in 2^16 at this stage.

Let's see.  For this to be the case, it would require that the
MAC_OPTIONAL bit is set in the Field Type, that's a 50% chance.  Then
the Field Length would have to match the actual packet length, that's a
1 in 2^16.  So I'm seeing this as a 1 in 2^17 chance.

But our numbers disagree, so what's the right answer?

But even if you are right and the answer is closer to 1 in 2^14, that
means that old software sending auth'd packets to a new server will see
a "false hit" on this once in every 16k packets.  In that case, if the
MAC_OPTIONAL bit is set, this one packet will be quietly accepted.  If
the packet was otherwise legitimate, it would have been accepted
anyway.  No big deal.  If the packet arrives and is rejected because it
had no auth, it will be logged (one can expect) and dropped.  That's 1
dropped packet in 16,384 packets.  I submit that's not a big deal.

If you are running an enterprise with 16k clients, that means that you
might see one of these "failures" every poll interval.

That will probably get noticed, and folks can either change a
configuration/ignore the problem (suboptimal) or upgrade their client
software.

If you are running 16 machines, that's 1 failure every 1k polls, or one
logged and dropped packet every 16 hours and 40 minutes.  If you're
running 1 server, you're looking at one of these events about once a
week.

Until you upgrade your software.

The above is with the simpler algorithm above.

With the more complicated algorithm (which might be LL(1) or LALR, not
sure and I don't feel like digging in to it), the numbers would drop a
bit, because we'd lean towards "success".  But it's not non-deterministic.

> to backtrack and try that one. And if that doesn't validate, you'll need

Huh?  A legacy MAC can only happen at a proper "boundary".  With the
more complicated algorith (that *could* be implemented pretty easily,
but I don't think it's worth it) will generate a set of potential
parsings, and it's a trivial loop to see which parsings are valid and
which are not, or we go with the simpler approach:

- parse/use valid EFs up to either:
- - the end of a LAST-EF
- - what comes next is not an EF
- then treat any remaining data as a legacy MAC

Yes, this approach will treat some small number of legacy MACs as EFs,
which will cause some fraction of those packets to be accepted, perhaps
when they should not be.

The primary place this might make a difference is with clients, as any
sort of peer/symmetric associations can easily save some state and we
can use I-DO there.

For clients, we COULD say that new client requests should include I-DO,
and if they don't we parse packets the old way, where the only EF we
expect is autokey.  If we don't want to apply any tests that see if that
the next data in the packet doesn't match a keyID that we're expecting
to help us decide what to do.

> You are assuming a MUST without documenting it.

I'm assuming a SHOULD.  If one does not do this, one gets to deal with
other problems.  That's a choice...

> actually decrypting.

But you MUST decrypt, because you MUST know if the auth succeeded or
not.  This isn't a new requirement, we've always had to do it.

> If you do insist on requiring data outside the

Huh?  I really don't see what you mean here.

> would be superior to the simpler choices.

Again, I'm not suggesting we require the complicated solution,  I'm
suggesting a simple solution.  And I'm suggesting we RECOMMEND it, not
REQUIRE it.

If somebody wants to implement one of the other solutions (that does not
require imposing length stuff on packet in the RFC), that's fine too.
Interoperability stuff will hash things out.

> re-enage and continue from there but until then, I am afraid I have

Are you still talking about the more complicated one that I've said we
should neither require nor implement?

H