Subject: [TLS] I-D Action: draft-ietf-tls-sni-encryption-08.txt

From: [email-masked]
Date: Mon, 07 Oct 2019 13:56:29 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Transport Layer Security WG of the IETF.

        Title           : Issues and Requirements for SNI Encryption in TLS
        Authors         : Christian Huitema
                          Eric Rescorla
	Filename        : draft-ietf-tls-sni-encryption-08.txt
	Pages           : 14
	Date            : 2019-10-07

Abstract:
   This draft describes the general problem of encrypting the Server
   Name Identification (SNI) TLS parameter.  The proposed solutions hide
   a Hidden Service behind a fronting service, only disclosing the SNI
   of the fronting service to external observers.  The draft lists known
   attacks against SNI encryption, discusses the current "co-tenancy
   fronting" solution, and presents requirements for future TLS layer
   solutions.

   In practice, it may well be that no solution can meet every
   requirement, and that practical solutions will have to make some
   compromises.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-tls-sni-encryption/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-tls-sni-encryption-08
https://datatracker.ietf.org/doc/html/draft-ietf-tls-sni-encryption-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-tls-sni-encryption-08

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Rob Sayre <[email-masked]>
Date: Tue, 8 Oct 2019 21:02:20 +0700

> Issues and Requirements for SNI Encryption in TLS

One issue not covered in this document is SNI encryption from CDNs to
Origin servers.

For example, if I use ESNI to make a request to Cloudflare, how does
Cloudflare then encrypt the SNI to the origin server?

It seems like this use case could be covered by allowing the SNI to be sent
alongside a client certificate (something many CDNs provide for). I think
an extension accompanying a client certificate would be encrypted, but
please correct me if I'm mistaken.

These CDN<->Origin connections can usually be served over IPv6, so having
the SNI in the ClientHello isn't necessarily as important.

thanks,
Rob

On Tue, Oct 8, 2019 at 3:56 AM <[email-masked]> wrote:

> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 8 Oct 2019 18:18:54 +0000

*   One issue not covered in this document is SNI encryption from CDNs to Origin servers.

I think deliberately so.  User-agent to origin (where sometimes the origin is a CDN or other intermediary) is the main case.

Akamai, for example, is not planning on doing ESNI to customer so far.

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 14:22:46 +0700

On Wed, Oct 9, 2019 at 1:19 AM Salz, Rich <[email-masked]> wrote:

>
>    - One issue not covered in this document is SNI encryption from CDNs
>    to Origin servers.
>
>
>
> I think deliberately so.  User-agent to origin (where sometimes the origin
> is a CDN or other intermediary) is the main case.
>

Well, TLS is a transport layer protocol. I think my suggestion might apply
to any client that's sending a certificate.

A link from CDN to Origin is just a particularly easy-to-deploy use case,
since client certificates are already in wide use and IPv6 tends to work
flawlessly.

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 9 Oct 2019 11:51:44 +0000

*   Well, TLS is a transport layer protocol. I think my suggestion might apply to any client that's sending a certificate.

I’m not disagreeing, I’m just explaining why it wasn’t a focus.

  *   A link from CDN to Origin is just a particularly easy-to-deploy use case, since client certificates are already in wide use and IPv6 tends to work flawlessly.

It does?  Gee, cool.

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 18:55:10 +0700

On Wed, Oct 9, 2019 at 6:51 PM Salz, Rich <[email-masked]> wrote:

>
>    - A link from CDN to Origin is just a particularly easy-to-deploy use
>    case, since client certificates are already in wide use and IPv6 tends to
>    work flawlessly.
>
>
>
> It does?  Gee, cool.
>

This response sounds like anger. I'm sorry I've caused you to feel angry.

It might be best to discuss technical concerns. Do you think an SNI field
sent with a client certificate is a bad idea? I'm not a cryptographer, so I
thought I would suggest the approach and see what people thought.

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 9 Oct 2019 12:31:28 +0000

It does?  Gee, cool.
I was being sarcastic, not angry. I shouldn’t have done so.

But now I am not sure what you are asking for.  Asking about CDN to Origin using ESNI or ESNI with a client cert?

---

From: Kyle Rose <[email-masked]>
Date: Wed, 9 Oct 2019 08:41:05 -0400

I'm struggling to understand the issue you're raising.

There's typically nothing special about the CDN to origin TLS interaction.
If the origin supports ESNI, why couldn't it advertise that? The CDN node
could just pick it up like any other TLS client would.*

The one way in which CDN to origin interactions may differ greatly from
client to origin interactions is the aggregation of many sessions to the
same origin inside the same connection, but this is at the application
layer. This aggregation may improve privacy by decoupling incoming traffic
from upstream requests through persistent connections and/or pre-cached
content.

Can you more precisely define your concern?

Kyle

*Not clear that it would be helpful, since the origin is probably obvious
from the destination IP, but I think the whole ESNI discussion presumes
traffic analysis is either hopelessly naïve or impossible, so I'll just
stipulate that and proceed from there.

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 19:47:06 +0700

> using ESNI or ESNI with a client cert?

I'm wondering what the backhaul traffic from CDN to Origin looks like, even
if a user-agent request to the CDN used ESNI. I noticed that many CDNs
provide client certificates.

In TLS handshakes that use a client certificate, it seems like the SNI
might be able to be sent with the second message from the client (alongside
the client certificate).

thanks,
Rob

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 9 Oct 2019 05:51:15 -0700

On Wed, Oct 9, 2019 at 5:47 AM Rob Sayre <[email-masked]> wrote:

>> But now I am not sure what you are asking for.  Asking about CDN to
>> Origin using ESNI or ESNI with a client cert?
>>
>
> I'm wondering what the backhaul traffic from CDN to Origin looks like,
> even if a user-agent request to the CDN used ESNI. I noticed that many CDNs

How would that work? The SNI is used by the server to determine what
certificate to send. That's why it's sent in CH.

-Ekr

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 19:54:53 +0700

> certificate to send. That's why it's sent in CH.

I understand why SNI is needed for IPv4 addresses.

But, if I have Cloudflare (or any CDN) configured for a domain, and the
origin is only available via IPv6, the need for a disambiguating SNI in the
ClientHello from CDN to Origin is not clear.

thanks,
Rob

---

From: Kyle Rose <[email-masked]>
Date: Wed, 9 Oct 2019 08:56:19 -0400

> provide client certificates.

Some origins do require client certificates, but not all. This is up to the
customer.

> might be able to be sent with the second message from the client (alongside

As I alluded to in the footnote from my last reply, I'm not sure how much
value this would have since the identity of the origin is typically evident
from the destination IP.

Kyle

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 9 Oct 2019 12:59:06 +0000

*   But, if I have Cloudflare (or any CDN) configured for a domain, and the origin is only available via IPv6, the need for a disambiguating SNI in the ClientHello from CDN to Origin is not clear.

That assumes that there is a one-to-one correspondence between an origin and its certificate, which isn’t true.  I might have “api.example.com” and “new-api.example.com” at the same IP address.

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 20:04:00 +0700

On Wed, Oct 9, 2019 at 7:59 PM Salz, Rich <[email-masked]> wrote:

>
>    - But, if I have Cloudflare (or any CDN) configured for a domain, and
>    the origin is only available via IPv6, the need for a disambiguating SNI in
>    the ClientHello from CDN to Origin is not clear.
>
>
>
> That assumes that there is a one-to-one correspondence between an origin
> and its certificate, which isn’t true.  I might have “api.example.com”
> and “new-api.example.com” at the same IP address.
>

I don't think that's quite what I'm proposing. I'm proposing (optionally)
sending the SNI with a client certificate. I agree that SNI in ClientHello
is needed to choose server certificates for IPv4, for the reason you say.

thanks,
Rob

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 9 Oct 2019 06:05:38 -0700

On Wed, Oct 9, 2019 at 6:04 AM Rob Sayre <[email-masked]> wrote:

> sending the SNI with a client certificate.
>

What are you trying to accomplish by doing that?

-Ekr

I agree that SNI in ClientHello is needed to choose server certificates for
> IPv4, for the reason you say.
>
> thanks,
> Rob
>

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 9 Oct 2019 13:09:24 +0000

*   I don't think that's quite what I'm proposing. I'm proposing (optionally) sending the SNI with a client certificate. I agree that SNI in ClientHello is needed to choose server certificates for IPv4, for the reason you say.

I didn’t say IPv4.  I might have all my “virtual origins” behind a single IP address, or a couple of them.

---

From: Paul Yang <[email-masked]>
Date: Wed, 9 Oct 2019 21:43:18 +0800

> On Oct 9, 2019, at 9:04 PM, Rob Sayre <[email-masked]> wrote:
> 
> On Wed, Oct 9, 2019 at 7:59 PM Salz, Rich <[email-masked] <mailto:[email-masked]>> wrote:
> But, if I have Cloudflare (or any CDN) configured for a domain, and the origin is only available via IPv6, the need for a disambiguating SNI in the ClientHello from CDN to Origin is not clear.
> 
> 
> That assumes that there is a one-to-one correspondence between an origin and its certificate, which isn’t true.  I might have “api.example.com <http://api.example.com/>” and “new-api.example.com <http://new-api.example.com/>” at the same IP address.
> 
> 
> I don't think that's quite what I'm proposing. I'm proposing (optionally) sending the SNI with a client certificate. I agree that SNI in ClientHello is needed to choose server certificates for IPv4, for the reason you say.

Are you suggesting: “In an IPv6 backend/origin scenario, the SNI should be sent along with client certificate instead of within ClientHello message”?

From my understandings, either IPv4 or IPv6 should have nothing to do with the concept “virtual host”, so a client (say, a CDN node) connects to either an IPv4/IPv6 server (say, an origin server), the SNI should applies the same in the TLS layer.

> Rob

Regards,

Paul Yang

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 20:46:34 +0700

On Wed, Oct 9, 2019 at 8:43 PM Paul Yang <[email-masked]> wrote:

>
> From my understandings, either IPv4 or IPv6 should have nothing to do with
> the concept “virtual host”
>

Hi Paul,

That is correct. However, the scarcity of IPv4 addresses is one major
factor driving the need for virtual hosts.

Thank you for reading the mailing list in detail.

thanks,
Rob

---

From: Paul Yang <[email-masked]>
Date: Wed, 9 Oct 2019 22:17:33 +0800

> On Oct 9, 2019, at 9:46 PM, Rob Sayre <[email-masked]> wrote:
> 
> On Wed, Oct 9, 2019 at 8:43 PM Paul Yang <[email-masked] <mailto:[email-masked]>> wrote:
> 
> From my understandings, either IPv4 or IPv6 should have nothing to do with the concept “virtual host”
> 
> Hi Paul,
> 
> That is correct. However, the scarcity of IPv4 addresses is one major factor driving the need for virtual hosts.

Yes, that’s right. So even IPv6 addresses are enormous enough to hold every domain name, we still can’t assume it’s all used in this way in practice. An administrator can always configure the origin server as hosting multiple domain names on one IPv6 address. It may not be very reasonable for doing so, but it could be done in that way. Actually popular web servers as NGINX supports such kind of configurations, for instance.

For TLS protocol, when being used between an IPv6 CDN node and an origin server, the SNI still need to be present in ClientHello to address the above circumstance; otherwise, the IPv6 origin may fail to choose the right host/certificate to finish the handshake.

Regards,

Paul Yang

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 21:20:10 +0700

On Wed, Oct 9, 2019 at 9:17 PM Paul Yang <[email-masked]> wrote:

> Yes, that’s right. So even IPv6 addresses are enormous enough to hold
> every domain name, we still can’t assume it’s all used in this way in
> practice. An administrator can always configure the origin server as
> hosting multiple domain names on one IPv6 address. It may not be very
> reasonable for doing so, but it could be done in that way. Actually popular
> web servers as NGINX supports such kind of configurations, for instance.
>
> For TLS protocol, when being used between an IPv6 CDN node and an origin
> server, the SNI still need to be present in ClientHello to address the
> above circumstance; otherwise, the IPv6 origin may fail to choose the right
> host/certificate to finish the handshake.
>

Hello,

I agree that it needs to be possible to include the SNI in ClientHello, but
not required.

thanks,
Rob

---

From: Christian Huitema <[email-masked]>
Date: Wed, 9 Oct 2019 08:17:19 -0700

I struggle a bit to understand how this thread is related to the SNI
Encryption Requirements document. Rob, you raise the question of using
and possibly hiding the SNI when using TLS to protect communication
between servers. That's a fine debate, but I don't see how this specific
usage of TLS and SNI affects the draft or change the requirements on SNI
encryption mechanics. We also have an interesting process issue, because
this draft already went through WG last call, IETF last call, and IESG
review. At this stage, it would take a huge issue to stop the presses
and rework the document.

-- Christian Huitema

On 10/9/2019 7:20 AM, Rob Sayre wrote:
>
>
> On Wed, Oct 9, 2019 at 9:17 PM Paul Yang <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>
>>     On Oct 9, 2019, at 9:46 PM, Rob Sayre <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>         From my understandings, either IPv4 or IPv6 should have
>>         nothing to do with the concept “virtual host”
>>
>>
>>     Hi Paul,
>>
>>     That is correct. However, the scarcity of IPv4 addresses is one
>>     major factor driving the need for virtual hosts.
>
>     Yes, that’s right. So even IPv6 addresses are enormous enough to
>     hold every domain name, we still can’t assume it’s all used in
>     this way in practice. An administrator can always configure the
>     origin server as hosting multiple domain names on one IPv6
>     address. It may not be very reasonable for doing so, but it could
>     be done in that way. Actually popular web servers as NGINX
>     supports such kind of configurations, for instance.
>
>     For TLS protocol, when being used between an IPv6 CDN node and an
>     origin server, the SNI still need to be present in ClientHello to
>     address the above circumstance; otherwise, the IPv6 origin may
>     fail to choose the right host/certificate to finish the handshake.
>
>
> Hello,
>
> I agree that it needs to be possible to include the SNI in
> ClientHello, but not required.
>
> thanks,
> Rob
>  
>

---

From: Rob Sayre <[email-masked]>
Date: Wed, 9 Oct 2019 22:22:19 +0700

On Wed, Oct 9, 2019 at 10:18 PM Christian Huitema <[email-masked]>
wrote:

> Encryption Requirements document. Rob, you raise the question of using and
> possibly hiding the SNI when using TLS to protect communication between
> servers. That's a fine debate, but I don't see how this specific usage of
> TLS and SNI affects the draft or change the requirements on SNI encryption
> mechanics. We also have an interesting process issue, because this draft
> already went through WG last call, IETF last call, and IESG review. At this
> stage, it would take a huge issue to stop the presses and rework the
> document.
>

Hello,

I didn't realize the document was at that phase. Also, my suggestion can be
added at any time.

So, I don't think this discussion should hold up any progression in the
IETF process.

Sorry for any confusion.

I support moving this document forward.

thanks,
Rob

---

From: Rob Sayre <[email-masked]>
Date: Thu, 10 Oct 2019 12:16:40 +0700

> What are you trying to accomplish by doing that?

I want to keep the SNI encrypted in TLS hops that use client certificates,
but where ESNI won't work.

For example, how is the SNI transmitted in the parens here:

[ Client ] -----> (ESNI) -----> [ CDN ] -----> (???) -----> [ Origin ]

I don't think a DNS-based solution like ESNI will work for that second hop,
because the origin tends to be identified by an IP address rather than a
domain name.

thanks,
Rob

---

From: Christian Huitema <[email-masked]>
Date: Wed, 9 Oct 2019 23:52:38 -0700

On 10/9/2019 10:16 PM, Rob Sayre wrote:
> On Wed, Oct 9, 2019 at 8:06 PM Eric Rescorla <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>         I don't think that's quite what I'm proposing. I'm proposing
>         (optionally) sending the SNI with a client certificate.
>
>
> I want to keep the SNI encrypted in TLS hops that use client
> certificates, but where ESNI won't work.
>
> I don't think a DNS-based solution like ESNI will work for that second
> hop, because the origin tends to be identified by an IP address rather
> than a domain name.

If the Origin is identified by IP address, an observer on path between
CDN and Origin just has to look at the IP address to find out whatever
information was in the SNI.

Do you have a specific threat in mind? It cannot be about just the CDN
and the Origin server, because observers can easily assess whether CDN X
serves origin O. Are you concerned that the observer will correlate
incoming traffic from client to CDN with outgoing traffic from CDN to
Origin?

This is an issue common to pretty much every relay service. I understand
that Tor has specific algorithms for dealing with that, involving for
example insertion of delays at the relay points. Is that the kind of
work that you have in mind? I am not sure that it would fit easily
within the TLS charter, but there are other working groups...

-- Christian Huitema

---

From: Rob Sayre <[email-masked]>
Date: Thu, 10 Oct 2019 13:59:40 +0700

On Thu, Oct 10, 2019 at 1:52 PM Christian Huitema <[email-masked]>
wrote:

> If the Origin is identified by IP address, an observer on path between CDN
> and Origin just has to look at the IP address to find out whatever
I don't think that is true for subdomains, and I also don't think it is
true if a client certificate is required (that would need to be checked
before routing traffic to an application).

Is there a sensible argument for leaving this traffic in the clear?

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 10 Oct 2019 15:53:14 +0000

*   I want to keep the SNI encrypted in TLS hops that use client certificates, but where ESNI won't work.

I have some questions about this, see below.

  *   For example, how is the SNI transmitted in the parens here:

  *   [ Client ] -----> (ESNI) -----> [ CDN ] -----> (???) -----> [ Origin ]

It is transmitted in the clear.  There is no architectural reason why it could not be ESNI.  But in my experience, there’s not much point in it, either.

What do you mean by client cert?  The CDN->Origin hop cannot present original Client’s certificate because (in general, maybe there are some exceptions) the CDN does not have the private key so it cannot do the necessary crypto operations.  That’s the right thing to do, otherwise anyone could present any client cert and claim to be the Client.  Instead, the Client certificate (or parts of it such as the subjectDN) are presented in newly-added HTTP headers.  The origin is configured to trust those headers, depending on the CDN/Origin relationship – it could be the CDN has it’s own client cert, it could be via IP filters, etc.

  *   I don't think a DNS-based solution like ESNI will work for that second hop, because the origin tends to be identified by an IP address rather than a domain name.

In our experience, the origin is identified by a DNS name.  I could double-check, but I don’t think *any* of our customer origins are identified by IP address.

---

From: Rob Sayre <[email-masked]>
Date: Thu, 10 Oct 2019 23:06:12 +0700

On Thu, Oct 10, 2019 at 10:53 PM Salz, Rich <[email-masked]> wrote:

>
>    - For example, how is the SNI transmitted in the parens here:
>
>
>
>    - [ Client ] -----> (ESNI) -----> [ CDN ] -----> (???) -----> [ Origin
>    ]
>
>
>
> It is transmitted in the clear.  There is no architectural reason why it
> could not be ESNI.  But in my experience, there’s not much point in it,
> either.
>

I am not sure I agree.

>
>
> What do you mean by client cert?  The CDN->Origin hop cannot present
> original Client’s certificate...
>

That's true. I was talking about something like this:
https://support.cloudflare.com/hc/en-us/articles/204899617-Authenticated-Origin-Pulls

Only the CDN is authenticated in that situation.

>
>
>
>    - I don't think a DNS-based solution like ESNI will work for that
>    second hop, because the origin tends to be identified by an IP address
>    rather than a domain name.
>
>
>
> In our experience, the origin is identified by a DNS name.  I could
> double-check, but I don’t think **any** of our customer origins are
> identified by IP address.
>

How does that work without introducing a CDN loop? Do you require the
origins to have obscure domain names? FWIW, the Cloudflare control panel
just has an IP address field. :)

thanks.
Rob

---

From: Watson Ladd <[email-masked]>
Date: Thu, 10 Oct 2019 09:07:04 -0700

On Thu, Oct 10, 2019, 8:54 AM Salz, Rich <[email-masked]> wrote:

>
>    - I want to keep the SNI encrypted in TLS hops that use client
> original Client’s certificate because (in general, maybe there are some
> exceptions) the CDN does not have the private key so it cannot do the
> necessary crypto operations.  That’s the right thing to do, otherwise
> anyone could present any client cert and claim to be the Client.  Instead,
> the Client certificate (or parts of it such as the subjectDN) are presented
> in newly-added HTTP headers.  The origin is configured to trust those
> headers, depending on the CDN/Origin relationship – it could be the CDN has
> it’s own client cert, it could be via IP filters, etc.
>
>
>

At least one customer of the CDN I work for  (namely my own website) uses
an IP address.

Shared hosting behind a CDN does exist where clients of the service
provider are signed up to the CDN, and it might be interesting  to use ESNI
there but the privacy risks are less extreme absent a global passive
adversary. Protecting client to shared infrastructure is what ESNI aims to
do.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 10 Oct 2019 16:14:57 +0000

*   At least one customer of the CDN I work for  (namely my own website) uses an IP address.

Sure, I get it.  Which is why I said “in our experience.” :)

  *   Shared hosting behind a CDN does exist where clients of the service provider are signed up to the CDN, and it might be interesting  to use ESNI there but the privacy risks are less extreme absent a global passive adversary. Protecting client to shared infrastructure is what ESNI aims to do.

The real point of this note:  strong +1 for this.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 10 Oct 2019 16:18:04 +0000

*   How does that work without introducing a CDN loop? Do you require the origins to have obscure domain names? FWIW, the Cloudflare control panel just has an IP address field. :)

It’s not obscure, it’s in DNS :)  But yes, the “true” origin has a different name.

One reason for using DNS is that big sites often use a multi-CDN load balancer.  They can shift in as little as 10-30 seconds.

---

From: Rob Sayre <[email-masked]>
Date: Thu, 10 Oct 2019 23:19:21 +0700

On Thu, Oct 10, 2019 at 11:15 PM Salz, Rich <[email-masked]> wrote:

>
>    - At least one customer of the CDN I work for  (namely my own website)
>    uses an IP address.
>
>
>
>    - Shared hosting behind a CDN does exist where clients of the service

Well, either way, it will be interesting to find out if insecure back haul
traffic becomes a point of competition. :)

thanks,
Rob

---

From: Rob Sayre <[email-masked]>
Date: Thu, 10 Oct 2019 23:44:06 +0700

M
On Thu, Oct 10, 2019 at 11:18 PM Salz, Rich <[email-masked]> wrote:

> One reason for using DNS is that big sites often use a multi-CDN load
> balancer.  They can shift in as little as 10-30 seconds.
>

Also, just to level-set the conversation, I am extremely familiar with this
routine (see sayrer.com)

It seems like load-balanced IPs with an API would do the job better than
DNS. At the time I had to deal with these problems, DNS was the only
choice, and every CDN vendor seemed incompetent and unreliable. So I can
understand why DNS is still the default.

thanks,
Rob

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 11:07:59 -0700

On Wed, Oct 9, 2019 at 10:16 PM Rob Sayre <[email-masked]> wrote:

> hop, because the origin tends to be identified by an IP address rather than
> a domain name.
>

I feel like we're perhaps talking past each other, so I'm going to start
back at the beginning.

In the ordinary case of Client -> Origin or Client -> CDN, the SNI
principally serves to tell the server which origin the client wants and
thus which certificate to serve. For this reason, it has to go in the CH.

In the case of CDN -> Origin, it seems like SNI could serve the same
purpose (say that the origin server is hosting multiple servers), in which
case it seems like the same requirements apply. OTOH, if the origin server
just hosts one origin, then you could potentially have the SNI delivered
later (as you suggest), but then why have SNI delivered at all?

What am I missing?
-Ekr

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 01:19:40 +0700

> thus which certificate to serve. For this reason, it has to go in the CH.

In general, I agree with you. But only because client certificates are
relatively rare for end users.

If a client certificate is required, the server probably can't send
application data in its first response.

> later (as you suggest), but then why have SNI delivered at all?

Two reasons:

1) the SNI could contain subdomain data, like "username.example.com"
2) if there is a client certificate, the SNI might be relevant in deciding
whether to accept it

I understand that the intermediary (CDN, etc) will be able to observe the
SNI, but it's not clear why it's ok that any passive observer to the origin
should be able to.

thanks,
Rob

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 11:45:00 -0700

> application data in its first response.

OK, I think we've now reached where we are talking past each other.

At a very high level, here's the TLS 1.3 handshake:

C->S: CH (w/ SNI)
S->C: SH, CERT, CV, FIN
C->S: [CERT, CV], FIN

In order for the server to send the certificate, which, as you can see, is
in its first flight, it has to have the SNI available. This means that it
has to go in the client's first flight. But the client's certificate is in
the client's second flight. And the client has to have the server's
certificate before it sends its certificate, because otherwise it can't
authenticate the server and so an active attacker can get the client's cert.

If you still disagree, maybe you could show me the ladder diagram you have
in mind?

-Ekr

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 01:59:32 +0700

On Fri, Oct 11, 2019 at 1:45 AM Eric Rescorla <[email-masked]> wrote:

> in its first flight, it has to have the SNI available.
>

I think this is the disconnect. The situation you describe is common
(sharing an IPv4 address), and I definitely understand why the SNI needs to
be in the ClientHello in that situation.

> This means that it has to go in the client's first flight. But the
> client's certificate is in the client's second flight. And the client has
> to have the server's certificate before it sends its certificate, because
> otherwise it can't authenticate the server and so an active attacker can
> get the client's cert.
>

The ladder diagram I have in mind is exactly as described in the TLS 1.3
RFC. Assume that the server can't send application data on its first
flight, and that the client must send a certificate on its second flight.

Then, assume that useful data can be transmitted in subdomain labels to a
server that has issued a wildcard certificate.

thanks,
Rob

---

From: Ben Schwartz <[email-masked]>
Date: Thu, 10 Oct 2019 16:57:02 -0400

Normally, virtual-hosted TLS servers are known to a client by their domain
name, and the client uses DNS to find an IP address corresponding to this
domain name.  The ESNI drafts are largely written in reference to this
configuration.

I think Rob is describing the case where a TLS client is contacting a
virtual-hosted origin that is defined by an IP address and an X.509 Common
Name, but for which it does not have a domain name.  In this case, reading
ESNI public keys out of the DNS is not an option, because there is no
domain name to look up.  This situation does seem to be reasonably common
for connections where the client is a CDN, and the server is an origin
backend.  (In that case, the domain name may exist, but its IP address and
ESNI keys will be the CDN's, not the backend's.)

The obvious solution is for the TLS client (i.e. the CDN) to support direct
entry of ESNI public keys alongside the IP address.  Users who want to be
able to rotate their ESNI keys more easily should use a backend identified
by a domain name that is distinct from the user-facing origin hostname.

I don't think there's any need to couple SNI encryption to use of client
certificates, nor is there any need to alter the ESNI protocol.

On Thu, Oct 10, 2019 at 3:00 PM Rob Sayre <[email-masked]> wrote:

>> In order for the server to send the certificate, which, as you can see,
>> is in its first flight, it has to have the SNI available.
>>
>
>> If you still disagree, maybe you could show me the ladder diagram you
>> have in mind?
>>
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 14:20:06 -0700

> server that has issued a wildcard certificate.

What are you anticipating the server would do with the SNI? Ordinarily we
expect the host header to be used for host identification purposes.

-Ekr

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 11 Oct 2019 09:36:19 +1100

On Fri, Oct 11, 2019, at 07:57, Ben Schwartz wrote:
> The obvious solution is for the TLS client (i.e. the CDN) to support 
> direct entry of ESNI public keys alongside the IP address. Users who 
> want to be able to rotate their ESNI keys more easily should use a 
> backend identified by a domain name that is distinct from the 
> user-facing origin hostname.

I was about to say the same thing.  No need to get fancy.

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 10:11:24 +0700

> I was about to say the same thing.  No need to get fancy.

Isn't that more complicated than sending the SNI in the second client
message, though?

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 03:28:10 +0000

>Isn't that more complicated than sending the SNI in the second client message, though?

The server needs to know which cert to use after it receives the *first* client message.

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 10:49:18 +0700

On Fri, Oct 11, 2019 at 10:28 AM Salz, Rich <[email-masked]> wrote:

> *>*Isn't that more complicated than sending the SNI in the second client
> message, though?
>
>
> The server needs to know which cert to use after it receives the **first**
> client message.
>

If the CDN ---> Origin traffic is IPv6, there's no need to serve multiple
certs from one IP address.

But, if the original request was for "username.example.com" to a CDN IPv4
address shared by many sites, my question is how "username.example.com"
would reach the origin and remain encrypted. I think a few people have
suggested uploading ESNI keys to the CDN, but it's not clear to me what
domain they would be for.

Maybe the best thing to do is just set up a site that documents whether the
CDN is sending SNI in the clear. I'm not really attached to any given
solution, but that will probably help them find one.

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 12:12:03 +0000

*   If the CDN ---> Origin traffic is IPv6, there's no need to serve multiple certs from one IP address.

Customers don’t want to be limited to that, for reasons I’ve tried to explain before.

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 11 Oct 2019 06:23:32 -0700

> Isn't that more complicated than sending the SNI in the second client

Well, both of these are more complicated than Host header. What's wrong
with that?

-Ekr

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 20:45:05 +0700

> Well, both of these are more complicated than Host header. What's wrong

The SNI and the host header often have to match (or at least have a known
mapping), because the origin server might want to prevent domain fronting.
This might be less of an issue over IPv6, but I think that's the reason the
two fields are currently more tightly-coupled than they were originally.

This issue has turned out to be surprisingly subtle, and current practices
seem to vary widely. My goal is to keep the SNI encrypted on the wire from
CDN to Origin (I understand that the SNI is visible to the CDN).

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 16:03:46 +0000

>The SNI and the host header often have to match (or at least have a known mapping), because the origin server might want to prevent domain fronting.

More likely the CDN.  Amazon and Google used to do this but stopped (https://www.theverge.com/2018/4/30/17304782/amazon-domain-fronting-google-discontinued).  I don’t see how the origin is involved.

> My goal is to keep the SNI encrypted on the wire from CDN to Origin (I understand that the SNI is visible to the CDN).

Use DNS entries for the origin then ESNI works.  If you follow your “IPv6 uniquely identifies the origin” assumption, then ESNI is pointless.

---

From: Rob Sayre <[email-masked]>
Date: Fri, 11 Oct 2019 23:57:14 +0700

On Fri, Oct 11, 2019 at 11:03 PM Salz, Rich <[email-masked]> wrote:

> *>*The SNI and the host header often have to match (or at least have a
> known mapping), because the origin server might want to prevent domain
> fronting.
>
>
>
> More likely the CDN.  Amazon and Google used to do this but stopped (
> https://www.theverge.com/2018/4/30/17304782/amazon-domain-fronting-google-discontinued).
> I don’t see how the origin is involved.
>
>
>
> > My goal is to keep the SNI encrypted on the wire from CDN to Origin (I
> understand that the SNI is visible to the CDN).
>
>
>
> Use DNS entries for the origin then ESNI works.  If you follow your “IPv6
> uniquely identifies the origin” assumption, then ESNI is pointless.
>

Hello,

I would like to reiterate the use case:

How does a request of the form "username.example.com" get through a CDN to
an Origin while leaving the SNI encrypted on the wire?

It sounds like you're saying the domain name should change from the CDN to
the Origin, but that doesn't seem like something that's automatically
supported or interoperable.

I also disagree with the argument that ESNI is pointless when “IPv6
uniquely identifies the origin”.

thanks,
Rob

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 17:11:08 +0000

*   How does a request of the form "username.example.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__username.example.com&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=4LM0GbR0h9Fvx86FtsKI-w&m=ujs6Lkbc_IGTiuLdcDk8syhWP1v9lNpztl9OxZuCvas&s=hBGHuzwfs66lIYRw2lkpneJu72vkeC9m5HH46EJ0i3c&e=>" get through a CDN to an Origin while leaving the SNI encrypted on the wire?

The CDN needs to see the decrypted SNI.  If the CDN and origin share the ESNI keys, the CDN can just pass the original ESNI value along.  If the CDN and origin do not share ESNI keys, then the CDN will have to re-encrypt.  If that is an issue you haven’t explained why or I missed.

  *   It sounds like you're saying the domain name should change from the CDN to the Origin, but that doesn't seem like something that's automatically supported or interoperable.

I guess it depends on the CDN.  I said that’s how my employer works, you said CloudFlare doesn’t work that way, and I didn’t quite understand what Watson said. :)

  *   I also disagree with the argument that ESNI is pointless when “IPv6 uniquely identifies the origin”.

Can you explain why?

---

From: Rob Sayre <[email-masked]>
Date: Sat, 12 Oct 2019 14:18:45 +0700

On Sat, Oct 12, 2019 at 12:11 AM Salz, Rich <[email-masked]> wrote:

>
>
>    - How does a request of the form "username.example.com
>    <https://urldefense.proofpoint.com/v2/url?u=http-3A__username.example.com&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=4LM0GbR0h9Fvx86FtsKI-w&m=ujs6Lkbc_IGTiuLdcDk8syhWP1v9lNpztl9OxZuCvas&s=hBGHuzwfs66lIYRw2lkpneJu72vkeC9m5HH46EJ0i3c&e=>"
>    get through a CDN to an Origin while leaving the SNI encrypted on the wire?
>
>
>
> The CDN needs to see the decrypted SNI.
>

Agreed.

> If the CDN and origin share the ESNI keys, the CDN can just pass the
> original ESNI value along.  If the CDN and origin do not share ESNI keys,
> then the CDN will have to re-encrypt.  If that is an issue you haven’t
> explained why or I missed.
>

It's definitely one approach. I am not sure which keys should be used for
the CDN -> Origin traffic, though. I suggested sending the SNI with the
client cert, because it seemed simpler if client certs are already being
used (an option I recommend for CDN -> Origin traffic).

EKR's Host header suggestion may also be viable, but my goal is actually to
make some TLS-level decisions about traffic based on the SNI. EKR's
proposal also assumes some level of Host header / SNI divergence is allowed
by the Origin host. These policies are not clear to me. For example, I'm
not sure what rules a Cloudflare to Google Cloud Platform link would need
to follow (this is a common enough setup that they have an egress
agreement).

>
>    - I also disagree with the argument that ESNI is pointless when “IPv6

I agree that a unique IPv6 address would expose the fact that a CDN is
communicating with an Origin, but I also think subdomain data could be used
for tracking, and so the SNI should still be encrypted.

thanks,
Rob

---

From: Patrick McManus <[email-masked]>
Date: Sat, 12 Oct 2019 10:09:41 -0400

some thoughts after catching up on this thread:

* cdn -> origin ime generally resolves via DNS for the same reason you want
anything else to resolve via DNS: a level of indirection is handy for
management. Occasionally it bypasses DNS for the same reasons you want
anything to bypass DNS: a level of indirection adds tail risk and overhead.

* origins sometimes have reasonable potential for ESNI anonymity sets
themselves .. cloud hosting environments would be good examples. this
should be encouraged.

* it seems pretty unusual that a CDN and Origin would share keys.. the
major use case of the CDN key is to cover an anonymity set that is a
significant number of distinct customers.

* using one v6 per origin (when you've got multiple origins available)
isn't a great pattern imo - not only does it make ESNI rather useless (with
an anonymity set of 1). but it fails to leverage the handshake and
congestion control amortizations across origins that h2/h3 can and will be
able to give you. We've built all the necessary http authority muxxing
stuff into higher levels with SNI and HTTP at this point, there's no need
to force it back into correlations withe the transport addresses.

* a few folks do like to authenticate the cdn to the origin with client
certs. That's nifty - but overall its pretty unpopular for the same reasons
managing distributed keys are always unpopular - its a hassle to manage in
the wild certs in a low risk way. DNS (scoped to ESNI - not as
authentication replacement) is a bit nicer in this respect because you
don't need to manage all the clients (multi cdn), but rather just a origin
property..

tldr; imo none of this works if the origin does not have a decent anonymity
set potential. If it does, just reuse esni for that hop rather than minting
something new.

On Sat, Oct 12, 2019 at 3:19 AM Rob Sayre <[email-masked]> wrote:

> EKR's Host header suggestion may also be viable, but my goal is actually
> to make some TLS-level decisions about traffic based on the SNI. EKR's

---

From: Rob Sayre <[email-masked]>
Date: Sat, 12 Oct 2019 22:37:39 +0700

On Sat, Oct 12, 2019 at 9:10 PM Patrick McManus <[email-masked]>
wrote:

>
> tldr; imo none of this works if the origin does not have a decent
> anonymity set potential. If it does, just reuse esni for that hop rather
> than minting something new.
>

Thank you for the thoughtful response. I think it might be helpful to use a
concrete example. The example I have in mind uses CDNs in two distinct ways.

Let's say it's an HTML document from "username.example.com" that looks
something like this:

<html>
 <body>
   <img src="[CDNHOST]/foo.jpg">
   <img src="[CDNHOST]/bar.jpg">
   <img src="[CDNHOST]/baz.jpg">
 </body>
</html>

One use case for CDNs is to serve those jpegs with a TTL. I've often seen
servers programmed to switch between CDNs in the HTML (or JSON, etc). So,
the ops team for "username.example.com" might have a switch that changes
"CDNHOST" from Level3 to Cogent (choose any of the big providers). This
provides agility that does not rely on any CDN provider, and doesn't rely
on DNS TTLs. That's good, because CDNs often encounter peering issues that
are not in their control. This experience might be biased toward social
media, where traffic is biased toward newer, popular content.

The other use case concerns the HTML document from "username.example.com".
This might be served with a zero-length TTL. This is done so that
personalized API traffic can be served over private backhaul links, rather
than the open Internet. Then, a Point-of-Presence data center will serve
the traffic as closely as possible to the client (I know Patrick knows all
of this, but I'm trying to be very clear for everyone).

Two points from Patrick's message are confusing to me:

- "using one v6 per origin (when you've got multiple origins available)
isn't a great pattern imo"

I meant to describe one IPv6 address per TLD+1 domain, so "
username1.example.com" and "username2.example.com" could be served from the
same IPv6 origin. It's not clear how to do this with an encrypted SNI from
CDN to origin. I understand that some people think ESNI keys are the way to
go, but I don't think anyone actually does this right now.

- "a few folks do like to authenticate the cdn to the origin with client
certs. That's nifty - but overall its pretty unpopular for the same reasons
managing distributed keys are always unpopular..."

It doesn't seem too complicated to me:

Maybe it is complicated on the CDN side, but not for the origin. And it
seems less complicated than uploading ESNI keys to the CDN.

thanks,
Rob