Subject: [Ntp] Wildcards in NTS certificate checking

From: Hal Murray <[email-masked]>
Date: Tue, 12 Apr 2022 23:10:23 -0700

Wildcards are deprecated by RFC 6125

https://datatracker.ietf.org/doc/draft-ietf-uta-rfc6125bis/
says:

   *  The only legal place for a certificate wildcard name is as the
      left-most component in a domain name.

   A technology MAY disallow the use of the wildcard character in DNS
   names.  If it does so, then the specification MUST state that
   wildcard certificates as defined in this document are not supported.

----------

RFC 8915 doesn't say anything about wildcards.  I don't remember any 
discussion and I didn't find any with a quick search.

If we were still working on the NTS specs, what, if anything, would we say 
about wildcards?

Is the security risk of a leftmost wildcard low enough that we should jump on 
the bandwagon?  (Is there a bandwagon?)

Are SANs (Server Alternate Names) a better alternative?

-- 
These are my opinions.  I hate spam.

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Apr 2022 09:15:21 +0200

>>> Hal Murray <[email-masked]> schrieb am 13.04.2022 um 08:10 in
Nachricht
<[email-masked]>:

> https://datatracker.ietf.org/doc/draft‑ietf‑uta‑rfc6125bis/ 
> says:
> 
>       left‑most component in a domain name.
> 
> ‑‑‑‑‑‑‑‑‑‑

Hi!

> discussion and I didn't find any with a quick search.

I think when a certificate us used to verify a server, then a certificate for
a whole (sub)domain is not what should be used.
For web servers where you have a server farm behind a load-balancer, such
certificates may make sense, but not for NTP IMHO.

> Are SANs (Server Alternate Names) a better alternative?

I think they are (when talking about one host entity).

Regards,
Ulrich

> ‑‑ 
> _______________________________________________
> ntp mailing list
> [email-masked] 
> https://www.ietf.org/mailman/listinfo/ntp

---

From: "Marco Davids (IETF IMAP)" <[email-masked]>
Date: Wed, 13 Apr 2022 13:50:02 +0200

Op 13-04-2022 om 08:10 schreef Hal Murray:

> about wildcards? >
> the bandwagon? 

In my view this is not only about security, but also about 
interoperability.

If the owner of a server decides that a (non partial) wildcard is good 
enough, why not accept this as a client?

Chrony does this.

NTPsec doesn't.

This leads to confusion.

The full discussion behind Hal's question can be found here btw:

https://gitlab.com/NTPsec/ntpsec/-/issues/729

-- 
Marco

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 13 Apr 2022 13:26:52 +0000

I think there's a bit of confusion.  6125, and 6125-bis (I'm primary author), talking about "*.example.com" in DNS names. The only place DNS names should show up is in SAN fields. Not in the "CN" field of the subjectDN.  They are widely used.  The upcoming draft tightens the rules a bit.

If there's a place I can add more text to avoid this confusion, please let me know.

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Apr 2022 15:39:08 +0200

Also there had been some "bad tricks" in certificates to fool matching the
names in the past.

 >>> "Salz, Rich" <rsalz=[email-masked]> schrieb am 13.04.2022 um
15:26
in Nachricht <[email-masked]>:
> I think there's a bit of confusion.  6125, and 6125‑bis (I'm primary
author), 
> talking about "*.example.com" in DNS names. The only place DNS names should

> show up is in SAN fields. Not in the "CN" field of the subjectDN.  They are

> widely used.  The upcoming draft tightens the rules a bit.
>

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 13 Apr 2022 13:48:47 +0000

names in the past.

Yes.  But that's the past.  Things like UTF8 substitution of "similar" looking characters, for example, is not a problem: DNS names, the only allowed name in a SAN field, don't have that issue.  Maybe there are others; if you know of things please let me know.

The next NTP should be compliant with 6125-bis.  I don't know why the WG would want to disallow wildcards -- being able to say "*.ntp.nist.gov" seems useful -- but if the WG wants to do that, it is our prerogative.

---

From: Hal Murray <[email-masked]>
Date: Wed, 13 Apr 2022 12:05:08 -0700

[email-masked] said:
>  I think there's a bit of confusion.  6125, and 6125-bis (I'm primary
> author), talking about "*.example.com" in DNS names. The only place DNS names
> should show up is in SAN fields. Not in the "CN" field of the subjectDN.
> They are widely used.  The upcoming draft tightens the rules a bit.

There is a good chance that I don't know enough about certificates to be 
asking the right questions.

6125 says deprecated.  I think 6125-bis is trying to allow wildcards on the 
left end.  That doesn't seem like "tightens" to me.

Could you please say more about "widely used"?

>From a security standpoint, how much risk does using wildcards add?  Are the 
potential risks of a bad guy doing something on a correctly setup system 
offset by the additional complexity of setting things up correctly without 
wildcards?  (or something like that)

> If there's a place I can add more text to avoid this confusion, please let me

I wasn't aware of the difference between CN and SAN and don't understand the 
significance yet.

This is what started the recent discussion.  

2022-02-22T19:32:06 ntpd[4932]: NTSc: certificate subject name: 
/C=NL/ST=Gelderland/O=Stichting Internet Domeinregistratie 
Nederland/CN=*.time.nl

Should that be OK?

Is there a simple example of using wildcards, with or without SAN?  I'm 
looking for the normal or typical case.

The case that I've used is:
  CN=ntp1.glypnod.com, SAN=ntp.glypnod.com (and same for ntp2)
DNS for ntp.example.com gives IP addresses for ntp1 and ntp2.
That takes N certificates, each with 1 SAN slot.

If I were doing that with wildcards, what would it look like?

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 13 Apr 2022 19:30:52 +0000

Fair enough.

6125 doesn't deprecate wildcards, it deprecates the use of the Common Name part of the SubjectName.  And if that's an area of confusion for you, it's not surprising that you're confused. :)  

>    Could you please say more about "widely used"?

Sure.  The world's most popular CA LetsEncrypt supports it, as do almost every other CA.  MANY hosts on the Internet use it, for example:
	; openssl s_client -connect www.cloudflare.com:443 </dev/null | openssl x509 -text
...cut...
            X509v3 Subject Alternative Name: 
                DNS:*.www.cloudflare.com, DNS:www.cloudflare.com
...cut  

The risk is that if your TLS private key is shared across multiple hosts, then it might be easier for an attacker to grab it.  MIGHT be: it all depends on your security infrastructure and operational processes. It also allows a rogue insider to PERHAPS act as something like "east-coast.www.cloudflare.com" when there is no such host, and fool users.

    significance yet.

A CN is part of the "Subject" distinguished name.  Using the above example:
       Subject: C=US, ST=California, L=San Francisco, O=Cloudflare, Inc., CN=www.cloudflare.com
It's just an unstructured (well mostly unstructured, there are some rules but not enough to rely on) name of who the certificate is *for* as opposed to the name of the CA that issued it.

The "CN" field should be *ignored.*  It's just a name, and there's no requirement that if something "looks like" a DNS name or wildcard, that it was intended to actually be one.

    /C=NL/ST=Gelderland/O=Stichting Internet Domeinregistratie 
    Nederland/CN=NL Time Authority

The code *might* want to log what the SAN field is as well.  Or just not bother.

Do the examples here help?

>    If I were doing that with wildcards, what would it look like?

It might be something like 'CN=Glpnod time service' and have a set of SAN fields that would display like
            X509v3 Subject Alternative Name: 
                DNS:ntp1.glypnod.com, DNS:ntp2.glypnod.com

You could use a single wildcard "DNS: *.glypnod.com", but the safest way to do that is put your NTP nodes into a separate subdomain and wildcard that:
	DNS:*.ntp.glypnod.com

Hope this helps.

>    These are my opinions.  I hate spam.

With mustard and pickles I'm told that it's tolerable. :)

---

From: "Marco Davids (IETF)" <[email-masked]>
Date: Thu, 14 Apr 2022 09:35:42 +0200

Op 13-04-22 om 21:30 schreef Salz, Rich:

>  It also allows a rogue insider to PERHAPS act as something like "east-coast.www.cloudflare.com" when there is no such host

It shouldn't be too hard for a roque insider that has the power to do 
this, to get a non-wildcard for that same fake 'east-cost' host either.

My point remains; if the owner of a server chooses to use a wildcard, 
maybe after carefully weighting and accepting the risks (i.e. by 
mitigating them via DNSSEC, CAA and RPKI), why should a client than 
choose to deny that? That's not for the client to decide, IMHO.

NTPsec is one of the very rare pieces of software that nevertheless 
refuses wildcards (in NTS client-mode) by default.

And by doing so it's deviating from the behaviour of other software 
(read: Chrony), thus causing confusion and interoperability issues. 
Ironically, NTPsec in NTS server-mode gladly accepts a wildcard, while 
refusing them in client-mode at the same time, so it's basically 
incompatible with itself. That doesn't make sense.

If only the RFC's would be more explicit on this matter, but 
unfortunately they are not. And as long as they aren't, the default 
behaviour should be to accept (non partial) wildcards, rather than 
refusing them.

That's the way I see it, at least.

-- 
Marco

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 14 Apr 2022 14:07:14 +0000

Agree.

Someone should file a bug. That's not compliant with common behavior, as you point out.

>   If only the RFC's would be more explicit on this matter, but 
    unfortunately they are not

Which RFC's?  I think 6125 is pretty clear about accepting wildcards, and the forthcoming update even more so.

---

From: Hal Murray <[email-masked]>
Date: Thu, 14 Apr 2022 09:39:27 -0700

> 6125 doesn't deprecate wildcards, it deprecates the use of the Common Name
> part of the SubjectName.  And if that's an area of confusion for you, it's
> not surprising that you're confused. :)

Thanks.

This doesn't use the word "deprecated", but it looks pretty strong to me:

7.2.  Wildcard Certificates

   This document states that the wildcard character '*' SHOULD NOT be
   included in presented identifiers but MAY be checked by application
   clients (mainly for the sake of backward compatibility with deployed
   infrastructure).  As a result, the rules provided in this document
   are more restrictive than the rules for many existing application
   technologies (such as those excerpted under Appendix B).  Several
   security considerations justify tightening the rules:

There are two uses of the word "deprecated" in 6125, pages 43 and 46.  That's 
off in appendix land.

------

The introduction to 6125bis says:

   *  Constrain wildcard certificates so that the wildcard can only be
      the left-most component of a domain name.

The end of section 3 says:

Is that section intended to apply retroactively to existing RFCs?

Overall, it seems to be retracting section 7.3 but it doesn't come right out and say it.

---------

NTS is (was) a new protocol.  We could have said no-wildcards.  On the other hand, we are dependent on an existing infrastructure to provide certificates and they support wildcards so some people are expect them to work for NTS.

I haven't found a good discussion on why one would want to use wildcards and/or the increased security risks of using them and/or the tradeoffs.  Is there a good paper or blog out there?

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 14 Apr 2022 17:06:49 +0000

I find the language of 6125 a little hard to get through at times. I find the fact that it doesn't use "deprecated" often is one of those reasons. 

Note that the section you quoted uses "SHOULD NOT" which says "don't unless you have a valid reason" and "MAY" which means optional. The updated version inverts this to be more clear and match common practice:
	A technology MAY disallow the use of the wildcard character in DNS names. If it does so, then the specification MUST state that wildcard certificates as defined in this document are not supported.

>    NTS is (was) a new protocol.  We could have said no-wildcards.  On the other hand, we are dependent on an existing infrastructure to provide certificates and they support wildcards so some peopl…

In many TLS stacks you have to explicitly turn off wildcard support when checking the certificate.  (E.g., in OpenSSL you have to explicitly turn on the X509_CHECK_FLAG_NO_WILDCARDS bit).

>    I haven't found a good discussion on why one would want to use wildcards and/or the increased security risks of using them and/or the tradeoffs.  Is there a good paper or blog out there?

I don't know of a single place that has that discussion. https://en.wikipedia.org/wiki/Wildcard_certificate is a short overview, and maybe some of its links are useful.

---

From: Hal Murray <[email-masked]>
Date: Thu, 14 Apr 2022 11:22:55 -0700

> My point remains; if the owner of a server chooses to use a wildcard, maybe
> after carefully weighting and accepting the risks (i.e. by mitigating them
> via DNSSEC, CAA and RPKI), why should a client than choose to deny that?
> That's not for the client to decide, IMHO. 

Why doesn't the client have input on that decision?

It seems just as reasonable to me for a client who wants the best security he 
can get to want to skip servers that aren't willing to setup no-wildcard 
certificates.

--------

The S in NTS is security.  Time is used in many security protocols.  I think 
it's important that we get it right.

Note that RFC 8915 came out in Sep 2020 so it's been well over a year before 
this issue came up.  What happened to make wildcards interesting?

--------

Is there a term for non-partial wildcards in the leftmost slot?  Things like 
*.example.com

I'll call them left-wildcards.

--------

The change you want is a one line edit.

What I'm missing is a discussion of the risks/benefits associated with 
left-wildcards and consensus from the NTS community that we should support 
them.

Or something similar from a security group saying the increased risk from 
left-wildcards is insignificant so they should be widely accecpted.  That's as 
compared to they are widely deployed so you should accept them for 
interoperability in spite of the increased insecurity.

---------

I can't tell if I'm being a nut-case for dragging my feet or a hero for 
starting a long overdue trend to get rid of wildcards.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 14 Apr 2022 19:08:27 +0000

>    Why doesn't the client have input on that decision?

The client is the ultimate authority, as it can examine the server certificate and decide not to trust it.

How do you know, for example, that *.nts.nist.gov isn't using replicated HSM's for example?

You also need to come up with something that is reasonably deployable.  After all, the most secure machine is one that is air-gapped and not on any network.  Not one that I'd want to use (

    *.example.com

No.  They're just called wildcard certificates.  You might want to have another term for "*www.example.com" but I would not bother.

Where do you want to find such consensus?  Supported by all browsers, and therefore not just HTTP but any other protocols that are built into the browser.  Supported by the CA/Browser Forum guidelines.

>    Or something similar from a security group saying the increased risk from 
    left-wildcards is insignificant so they should be widely accecpted

You could ask the TLS working group. Perhaps better would be to ask the UTA working group.  If you're not on their mailing lists, you can still send email but the WG chair will have to approve it.

If nothing else, Marco pointed out an internal inconsistency that might need to be addressed, perhaps by documentation.

---

From: "Marco Davids (IETF)" <[email-masked]>
Date: Fri, 15 Apr 2022 09:17:01 +0200

> Which RFC's?  I think 6125 is pretty clear about accepting wildcards, and the forthcoming update even more so.

Sorry. What I meant to say is that 6125bis has this:

"A technology MAY disallow the use of the wildcard character in DNS
  names.  If it does so, then the specification MUST state that
  wildcard certificates as defined in this document are not supported."

And RFC8915 came out before that, so it hasn't got that statement.

Hence, we are a bit in limbo here. For me there's no doubt: NTS should 
allow wildcards, unless an update it published that says otherwise. But 
opinions differ on this

-- 
Marco

---

From: "Marco Davids (IETF)" <[email-masked]>
Date: Fri, 15 Apr 2022 09:20:53 +0200

Op 14-04-22 om 19:06 schreef Salz, Rich:

> In many TLS stacks you have to explicitly turn off wildcard support when checking the certificate. 

That's precisely what NTPsec decided to do:

https://gitlab.com/NTPsec/ntpsec/-/blob/master/ntpd/nts_client.c#L407

-- 
Marco

---

From: "Marco Davids (IETF)" <[email-masked]>
Date: Fri, 15 Apr 2022 09:43:01 +0200

> Why doesn't the client have input on that decision?

This has everything to do with interoperability and 'operational 
hygiene'. (or, if you your are still into it, Jon Postel's 'robustness 
principle'). Or, if you favour that one, like me, RFC1925, rule nr. 1.

> can get to want to skip servers that aren't willing to setup no-wildcard

Isn't that the job of the operator who deploys the software and picks 
the NTS servers to be used? If no wildcards are desired, than just don't 
configure any such NTS servers in the ntp.conf.

> this issue came up.  What happened to make wildcards interesting?

Good point. NTS has been out for a while now, the uptake is still low 
and we want it to be deployed more widely. The last thing we need are 
frustrated users that put it aside because it doesn't work. NTPsec and 
Chrony and all other pieces of software should work flawlessly with each 
other. Any obstacle that is in the way should be avoided when possible.

If wildcards are a problem for NTPsec, than perhaps a configuration 
option can be introduced for this, but the default setting should be to 
accept them, at least in my mind.

> I'll call them left-wildcards.

I call them non-partial. So ab*.example.com is a partial wildcard. The 
are truly deprecated, and *.example.com is a non-partial one, in my 
terminology anyways.

> left-wildcards and consensus from the NTS community that we should support

Yes, I agree. Well, I already gave my input, so there's little to add 
there. But to sum them up:

- wildcards are still common practice, and widely deployed and issued,
- it's not only about security, but also about interoperability,
- the default should be to accept them, because the RFC8915 doesn't say 
otherwise,
- perhaps make a configuration option in the software that disallows them

> left-wildcards is insignificant so they should be widely accecpted.

But they probably already are ;-) Any Chrony-user will confirm that. 
Probably other NTS-clients support them too (but I haven't verified that).

> starting a long overdue trend to get rid of wildcards.

Your cause is noble without a doubt. But as a guy with an operational 
background I learned that noble causes don't always lead to desired 
results. The last thing we need is that we are frustrating the process 
of broader NTS deployment.

-- 
Marco

---

From: Hal Murray <[email-masked]>
Date: Fri, 15 Apr 2022 13:32:42 -0700

mdavids=[email-masked] said:
> This has everything to do with interoperability and 'operational  hygiene'.
> (or, if you your are still into it, Jon Postel's 'robustness  principle').
> Or, if you favour that one, like me, RFC1925, rule nr. 1. 

Postel's robustness principle came out a long time ago when we were struggling 
to make basic things work.  Nobody had paid much attention to security yet.  
It would be a fun discussion to turn the clock back to that era and ask "What 
about security?"

> Isn't that the job of the operator who deploys the software and picks the
> NTS servers to be used? If no wildcards are desired, than just don't

Do server operators advertise that they do/don't use wildcards?  Do they ever 
change their minds?

> background I learned that noble causes don't always lead to desired  results.
> The last thing we need is that we are frustrating the process  of broader NTS
> deployment. 

Another last thing we need is to get a reputation for not being secure.

-------

Slight topic change, shifting to deployment...

Where are the NTS servers using wildcards?
Did something happen recently to trigger this discussion?

Is there NTS deployment activity or discussion that I've missed?

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 16 Apr 2022 13:58:18 +0000

change their minds?

There is no way to find out until the client has the cert and sees a "*" in the SAN field.  I am sure people have changed their minds about whether they use wildcards or not; the Internet is a big place.

>    Another last thing we need is to get a reputation for not being secure.

You will not get a reputation for being insecure if you support wildcards.  I mean, c'mon, Marco has pointed out that the software is inconsistent and I've pointed out that it is really common practice and that your policy decisions seem based on lack of knowledge. I've also suggested other places you could go to (the UTA and TLS working groups).  What more do you want?  I'm can’t imagine how else I can useful contribute to this conversation.

>    Where are the NTS servers using wildcards?

It's early days.

>    Did something happen recently to trigger this discussion?

I believe it started with Marco's first post on this thread.

>    Is there NTS deployment activity or discussion that I've missed?

Not that I am aware of.  But with any luck, there are organizations planning NTS deployments that will never get mentioned here.

---

From: Paul Gear <[email-masked]>
Date: Sun, 17 Apr 2022 11:42:07 +1000

On 15/4/22 17:43, Marco Davids (IETF) wrote:
> ...
> - the default should be to accept them, because the RFC8915 doesn't 
> say otherwise,

Hi everyone,

As a member of the NTP pool operator community, I plead with you: DO NOT 
ban the use of wildcards in NTS.  By all means, if you think that they 
are less secure (I think that is very debatable, but here is not the 
place for doing so), then provide your clients with the ability to 
discard associations with servers which use them.  But please do not 
make decisions in the standard which should be local policy decisions.  
Wildcards are a widely-expected feature of TLS implementations, and 
there are common scenarios in which they are the only means to achieve a 
desired configuration.

Regards,
Paul

---

From: "Marco Davids (IETF)" <[email-masked]>
Date: Mon, 18 Apr 2022 09:11:27 +0200

> Did something happen recently to trigger this discussion?

I used them on ntppool1.time.nl and ntppool2.time.nl until I found out 
(the hard way) that NTPsec as a client doesn't like them (while Chrony 
does).

Currently I have test server running on nts.time.nl with a wildcard. 
Ironically it is an NTPSec server.

-- 
Marco