Subject: [Acme] RFC draft-ietf-acme-acme-02 - tls SNI name

From: Tim Small <[email-masked]>
Date: Wed, 12 Oct 2016 11:45:13 +0100

Hello,

Having read draft-ietf-acme-acme-02, and made a number of early
deployments using the tls-sni-01 mechanism, I thought I'd highlight what
I perceive to be a possible shortcoming with the tls-sni-02 draft.

It's becoming more wide-spread (as a result of IPv4 exhaustion) to
deploy TLS SNI "switches" to provide IPv4 connectivity to autonomously
managed web servers.

The typical deployment pattern is that a web server is assigned a
publicly accessible IPv6 address only - this can be used for direct
management and direct IPv6 https requests etc.  The TLS request are
terminated on the web server itself.

The customer is also informed of an IPv4 addresses to be used for web
access by IPv4-only clients, and asked to provide a list of FQDNs which
they will serve.  IPv4 https requests are answered by proxy software,
which direct the IPv4 https request to the correct web server's IPv6
address (or an RFC-1918 IPv4 address etc.) based on SNI.

e.g. this haproxy example configuration fragment:

use-server customer1-staging if { req.ssl_sni -m beg
staging.customer1-example.com }
use-server customer2-live if { req.ssl_sni -m beg
staging.customer2-example.com }

I believe this will become increasingly common-place during the
transition to IPv6, and the current form of tls-sni-02 makes it hard to
use in this sort of environment.

The problem comes when using tls-sni-02 ACME challenges:

"The client MUST ensure that the certificate is served to TLS
connections specifying a Server Name Indication (SNI) value of SAN A."

i.e. the SNI string which the proxy sees during the challenge is always
"SAN A" (x.y.token.acme.invalid), for any server for which it acts as
the "https switch" for.

... so the proxy has no way of knowing which server the challenge
request should be directed to.  Some mechanism by which the proxy can
direct the request would be useful.

One which may provide an easy route for the service providers offering
this sort of setup (many would require no changes to their existing
configuration at all) would be to append the FQDN in the TLS SNI value
e.g. x.y.token.acme.client2-example.com

The client in question may have limited or no influence on the
configuration of the SNI proxy, so the more straight-forward the proxy
configuration, the-better.

For another example of a similar use-case with the same issue:

https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/

HTH,

Tim.

---

From: Daniel McCarney <[email-masked]>
Date: Thu, 13 Oct 2016 11:00:12 -0400

Hi Tim,

Thanks for sharing your thoughts. This was the first time I've heard about
this
sort of SNI multiplexing middleware in the context of IPv6 hosting.

I'm newer to the list/draft process but I suspect your idea of using
"x.y.token.acme.client2-example.com" to support the SNI multiplexing
decision
will run counter to the intent behind using the ".acme.invalid" suffix as
described in the current draft. Perhaps someone with more historical
context can
help clarify why that particular suffix strategy was selected.

It would help me understand your feedback better if you could speak to why
it
doesn't make more sense for users of this sort of middleware to favour the
HTTP-01 or DNS-01 challenge type. My own opinion leans towards thinking that
using a more appropriate challenge type is preferred to trying to address
TLS
middleware in the context of the TLS-SNI-02 challenge design. This class of
middleware can introduce all sorts of complication that might end up
significantly complicating the challenge design.

- Daniel/cpu

On Wed, Oct 12, 2016 at 6:45 AM, Tim Small <[email-masked]> wrote:

> _______________________________________________
> Acme mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/acme
>

---

From: Ben Irving <[email-masked]>
Date: Thu, 13 Oct 2016 16:39:39 +0000

Hello,

I have ran into a very similar use case. In my case I'm using Haproxy to
route tcp requests based on the server name indication to upstream web
servers where the TLS request is terminated. The ACME client is also
running on these upstream web servers. I'm forced to use HTTP-01 challenges
and am forced to open up port 80 as well on these upstream servers, which I
don't want to do. I don't expect the protocol to solve my particular use
case, but I'm also curious as to why acme.invalid was used as the suffix
for all TLS-SNI based challenges?

- Ben

---

From: James Kasten <[email-masked]>
Date: Thu, 13 Oct 2016 11:21:23 -0700

There are a few attack scenarios that "acme.invalid" attempts to account
for.  It protects sites like blogs that may allow users to select
subdomains and upload certificates (x.y.token.blog.com would be potentially
vulnerable).

Multiple virtual hosts on shared hosting environments are also a concern.
Using "acme.invalid" gives a clear indicator to the provider that the
certificate is intended for a challenge. "invalid" is a reserved TLD.
https://tools.ietf.org/html/rfc2606#section-2

In response to Tim.

I agree that including the domain within the challenge would be beneficial
for many deployments. The format of "x.y.token.domain.acme.invalid" has
come up in the past.

The idea was raised to the group at IETF 94, but was rejected for concern
of enterprise deployments going over the DNS byte limit. (DNS names are
restricted to a length of 253)
*https://github.com/ietf-wg-acme/acme/issues/23
<https://github.com/ietf-wg-acme/acme/issues/23>*
http://ietf.org/proceedings/94/minutes/minutes-94-acme

To be fair, the DNS challenge suffers from the same problem with the length
of the DNS entry, though to a lesser degree.

One of my colleagues, Nick Felt, recently ran a censys.io query to see how
many of the existing domains contained within certificates would be unable
to solve SNI challenges formatted like "x.y.token.domain.acme.invalid". He
found that the length restriction would have virtually no impact on the
existing publicly visible ecosystem.  There were only 10 domains over 170
characters long.

Perhaps including the domain should be reconsidered.

Best,
James

On Thu, Oct 13, 2016 at 9:39 AM, Ben Irving <[email-masked]> wrote:

>> Thanks for sharing your thoughts. This was the first time I've heard
>> about this
>> It would help me understand your feedback better if you could speak to
>> why it

---

From: Ilari Liusvaara <[email-masked]>
Date: Thu, 13 Oct 2016 21:58:25 +0300

> for all TLS-SNI based challenges?

One possible way to work around these sort of issues would be allowing
programmatic runtime routing control for the TLS demuxer.

And then have proxy service that validates the submitted CSRs and then
tries to obtain certificates for those.

(Allowing webservers to directly set routes would allow them to obtain
certs for other domains hosted behind the same loadbalancer.)

-Ilari

---

From: Alan Doherty <[email-masked]>
Date: Thu, 13 Oct 2016 21:10:38 +0100

At 17:39 13/10/2016  Thursday, Ben Irving wrote:
>Hello,
>
>I have ran into a very similar use case. In my case I'm using Haproxy to route tcp requests based on the server name indication to upstream web servers where the TLS request is terminated. The ACME client is also running on these upstream web servers. I'm forced to use HTTP-01 challenges and am forced to open up port 80 as well on these upstream servers, which I don't want to do. I don't expect the protocol to solve my particular use case, but I'm also curious as to why acme.invalid was used as the suffix for all TLS-SNI based challenges?

you shouldnt have to open port 80 to upstreams
(assuming on the haproxy you do the normal http redirecting on port 80  aka requests for http://whatever get a 301 redirect to https://same-domain/same-path

as the http-01 challenge will happily follow redirects and thus find/connect to https://whatever-domain/acme-specific-url

>- BenÂ 
>
>
>On Thu, Oct 13, 2016 at 8:00 AM Daniel McCarney <<mailto:[email-masked]>[email-masked]> wrote:
>Hi Tim,
>
>Thanks for sharing your thoughts. This was the first time I've heard about this
>"<http://x.y.token.acme.client2-example.com>x.y.token.acme.client2-example.com" to support the SNI multiplexing decision
>described in the current draft. Perhaps someone with more historical context can
>It would help me understand your feedback better if you could speak to why it
>using a more appropriate challenge type is preferred to trying to address TLS
>On Wed, Oct 12, 2016 at 6:45 AM, Tim Small <<mailto:[email-masked]>[email-masked]> wrote:
>Hello,
>
>management and direct IPv6 https requests etc.Â  The TLS request are
>they will serve.Â  IPv4 https requests are answered by proxy software,
><http://staging.customer1-example.com>staging.customer1-example.com }
><http://staging.customer2-example.com>staging.customer2-example.com }
>
>
>request should be directed to.Â  Some mechanism by which the proxy can
>e.g. <http://x.y.token.acme.client2-example.com>x.y.token.acme.client2-example.com
>
><https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/
>
>HTH,
>
>Tim.
>
><mailto:[email-masked]>[email-masked]
><mailto:[email-masked]>[email-masked]

---

From: Hugo Landau <[email-masked]>
Date: Fri, 14 Oct 2016 05:07:11 +0100

Perhaps clients should be allowed to specify a 'routing key' when
requesting that TLS-SNI validation be performed.

The name used would be:

  x.y.ROUTING-KEY.token.acme.invalid.

The routing key would be limited in length, up to say 16-32 bytes and
must be a single DNS label in [a-z0-9-]{1,63}.

If a routing key isn't specified, a routing key of "default" can be used.

Hugo Landau

---

From: Ben Irving <[email-masked]>
Date: Fri, 14 Oct 2016 15:31:59 +0000

On Thu, Oct 13, 2016 at 1:10 PM Alan Doherty <[email-masked]> wrote:

> (assuming on the haproxy you do the normal http redirecting on port 80
> aka requests for http://whatever get a 301 redirect to
> https://same-domain/same-path

Maybe I'm misunderstanding, but I don't think there is an ACME client that
runs on port 443 for challenge type http-01. The major hurdle here is that
I'm running the ACME client upstream. However, since I'm not able to route
the TLS-SNI challenges to the correct upstream client, I'm forced to use
HTTP-01 and open up port 80.

>
>
> as the http-01 challenge will happily follow redirects and thus
> find/connect to https://whatever-domain/acme-specific-url
>
>
> >- BenÂ
> >
> >
> >On Thu, Oct 13, 2016 at 8:00 AM Daniel McCarney <<mailto:
> [email-masked]>[email-masked]> wrote:
> >Hi Tim,
> >
> >"<http://x.y.token.acme.client2-example.com>
> x.y.token.acme.client2-example.com" to support the SNI multiplexing
> decision
> ><https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>

---

From: Alan Doherty <[email-masked]>
Date: Fri, 14 Oct 2016 17:53:22 +0100

At 16:31 14/10/2016  Friday, Ben Irving wrote:

>On Thu, Oct 13, 2016 at 1:10 PM Alan Doherty <<mailto:[email-masked]>[email-masked]> wrote:
>At 17:39 13/10/2016Â  Thursday, Ben Irving wrote:
>>Hello,
>>
>(assuming on the haproxy you do the normal http redirecting on port 80Â  aka requests for <http://whatever>http://whatever get a 301 redirect to <https://same-domain/same-path>https://same-domain/same-path
>
>Maybe I'm misunderstanding, but I don't think there is an ACME client that runs on port 443 for challenge type http-01. The major hurdle here is that I'm running the ACME client upstream. However, since I'm not able to route the TLS-SNI challenges to the correct upstream client, I'm forced to use HTTP-01 and open up port 80.Â 

your misunderstanding i said http-01 does not require opening port 80 to upstreams

http-01 will follow redirects 

thus if your proxy is directing www.domain1.com:443 at the upstream correctly

and 301 redirecting http://www.domain1.com:80 to https://www.domain1.com:443
which would be normal in most setups

http-01 will work fine without any use of port 80 on upstream server

(but on further thinking since)

(though if doing proxy why on earth would the upstream need anything other than a self signed cert (as its cert is never seen by public)

and just have the proxy run the acme client and answer rather than forward 
/.well-known/acme-challenge/
and update its own publicly visible cert only

btw in http-01 the acme client can specify to the server whether to use http://www.domain1.com/.well-known/acme-challenge/
or https://www.domain1.com/.well-known/acme-challenge/
directly

"The client’s response to this challenge indicates whether it would prefer for the validation request to be sent over TLS:

type (required, string):  
The string “simpleHttp” 
tls (optional, boolean, default true):  
If this attribute is present and set to “false”, the server will perform its validation check over unencrypted HTTP (on port 80) rather than over HTTPS. Otherwise the check will be done over HTTPS, on port 443." 
https://letsencrypt.github.io/acme-spec/#rfc.section.7.1

>Â 
>
>
>
>
>as the http-01 challenge will happily follow redirects and thus find/connect to <https://whatever-domain/acme-specific-url>https://whatever-domain/acme-specific-url
>
>
>>- BenÃ‚
>>
>>
>>On Thu, Oct 13, 2016 at 8:00 AM Daniel McCarney <<mailto:[email-masked]><mailto:[email-masked]>[email-masked]> wrote:
>>Hi Tim,
>>
>>"<<http://x.y.token.acme.client2-example.com>http://x.y.token.acme.client2-example.com><http://x.y.token.acme.client2-example.com>x.y.token.acme.client2-example.com" to support the SNI multiplexing decision
>>management and direct IPv6 https requests etc.Ã‚Â  The TLS request are
>>they will serve.Ã‚Â  IPv4 https requests are answered by proxy software,
>><<http://staging.customer1-example.com>http://staging.customer1-example.com><http://staging.customer1-example.com>staging.customer1-example.com }
>><<http://staging.customer2-example.com>http://staging.customer2-example.com><http://staging.customer2-example.com>staging.customer2-example.com }
>>
>>
>>request should be directed to.Ã‚Â  Some mechanism by which the proxy can
>>e.g. <<http://x.y.token.acme.client2-example.com>http://x.y.token.acme.client2-example.com><http://x.y.token.acme.client2-example.com>x.y.token.acme.client2-example.com
>>
>><<https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/
>>
>>HTH,
>>
>>Tim.
>>
>><mailto:[email-masked]><mailto:[email-masked]>[email-masked]
>><mailto:[email-masked]><mailto:[email-masked]>[email-masked]

---

From: Ben Irving <[email-masked]>
Date: Fri, 14 Oct 2016 17:07:22 +0000

On Fri, Oct 14, 2016 at 9:53 AM Alan Doherty <[email-masked]> wrote:

> >On Thu, Oct 13, 2016 at 1:10 PM Alan Doherty <<mailto:
> [email-masked]>[email-masked]> wrote:
> >(assuming on the haproxy you do the normal http redirecting on port 80Â
> aka requests for <http://whatever>http://whatever get a 301 redirect to <
> https://same-domain/same-path>https://same-domain/same-path
> >
> >Maybe I'm misunderstanding, but I don't think there is an ACME client
> that runs on port 443 for challenge type http-01. The major hurdle here is
> that I'm running the ACME client upstream. However, since I'm not able to
> route the TLS-SNI challenges to the correct upstream client, I'm forced to
> use HTTP-01 and open up port 80.Â
>
Really? That's great news.

> (though if doing proxy why on earth would the upstream need anything other
> than a self signed cert (as its cert is never seen by public)

Picture this, I have a host running 100 docker containers, each exposing a
single web application through an ephemeral port. Inside each of these
docker environments is the ACME client running with Nginx which is
reverse-proxy to a web application of any type. Haproxy is installed on the
host as is using TLS-SNI based routing to route TCP requests to the proper
docker container.

> btw in http-01 the acme client can specify to the server whether to use
> http://www.domain1.com/.well-known/acme-challenge/
won't I need a certificate then? I'm calling the client before nginx has
started.

>
> "The client’s response to this challenge indicates whether it would prefer
> for the validation request to be sent over TLS:
>
> type (required, string):
> The string “simpleHttp”
> If this attribute is present and set to “false”, the server will perform
> its validation check over unencrypted HTTP (on port 80) rather than over
> HTTPS. Otherwise the check will be done over HTTPS, on port 443."
> find/connect to <https://whatever-domain/acme-specific-url>
> https://whatever-domain/acme-specific-url
> >
> >
> >>- BenÃ‚
> >>
> >>
> [email-masked]><mailto:[email-masked]>[email-masked]>
> wrote:
> >>Hi Tim,
> >>
> >>"<<http://x.y.token.acme.client2-example.com>
> http://x.y.token.acme.client2-example.com><
> http://x.y.token.acme.client2-example.com>
> >><<http://staging.customer1-example.com>
> http://staging.customer1-example.com><http://staging.customer1-example.com
> >><<http://staging.customer2-example.com>
> http://staging.customer2-example.com><http://staging.customer2-example.com
> >>e.g. <<http://x.y.token.acme.client2-example.com>
> http://x.y.token.acme.client2-example.com><
> http://x.y.token.acme.client2-example.com>
> x.y.token.acme.client2-example.com
> >>
> >><<https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>
> https://scriptthe.net/2015/02/08/pass-through-ssl-with-haproxy/>

---

From: Ilari Liusvaara <[email-masked]>
Date: Fri, 14 Oct 2016 20:41:12 +0300

> won't I need a certificate then? I'm calling the client before nginx has

Yes, you need a self-signed certificate to bootstrap the system. And
AFAIK, none of the usual ACME clients support this kind of operation.

Standalone mode won't work because it assumes TLS-SNI (which won't work
with TLS) with TLS and it would break on renewal anyway.

-Ilari

---

From: Patrick Figel <[email-masked]>
Date: Fri, 14 Oct 2016 21:03:34 +0200

On 14/10/16 18:53, Alan Doherty wrote:
> btw in http-01 the acme client can specify to the server whether to 
> use http://www.domain1.com/.well-known/acme-challenge/ or 
> https://www.domain1.com/.well-known/acme-challenge/ directly
> 
> "The client’s response to this challenge indicates whether it would 
> prefer for the validation request to be sent over TLS:
> 
> type (required, string): The string “simpleHttp”
> tls (optional, boolean, default true): If this attribute is present
> and set to “false”, the server will perform its validation check over
>  unencrypted HTTP (on port 80) rather than over HTTPS. Otherwise the
>  check will be done over HTTPS, on port 443." 

Note that this is referring to the simpleHttp challenge type, which was
removed quite some time ago, due to possible domain validation
vulnerabilities with common server configurations[1]. The replacement -
http-01 - does not support such a "tls" parameter, and validation
requests are always performed via HTTP (though a redirect to https:// is
still accepted).

[1]: https://www.ietf.org/mail-archive/web/acme/current/msg00524.html