Subject: Re: [TLS] Two Multi-CDN proposals

From: Christopher Wood <[email-masked]>
Date: Fri, 1 Mar 2019 08:57:23 -0800

On Wed, Feb 27, 2019 at 11:34 PM Kazuho Oku <[email-masked]> wrote:
>
> Hi Chris,
>
> Thank you for writing down the PRs describing possible designs that we
> might adopt. I think it helps a lot in understanding the details and
> making accurate comparisons.
>
> My comments inline.
>
> 2019年2月27日(水) 8:19 Christopher Wood <[email-masked]>:
> >
> > Hi folks,
> >
> > Below are two PRs that seek to address the multi-CDN issue discussed
> > on this list and in meetings:
> >
> >    1. https://github.com/tlswg/draft-ietf-tls-esni/pull/136
> >    2. https://github.com/tlswg/draft-ietf-tls-esni/pull/137
> >
> > #136 implements the combined or stapled record approach discussed
> > several times, most recently in [1]. It includes these via an ESNIKeys
> > extension.
>
> Reading the PR, I think that there is a mild privacy concern. The
> issue with the unified record proposal is that it incentivizes the
> operators to create more ESNI records possibly using different keys,
> which is against the design principle of ESNI: provide anonymity set
> by using the least number of keys.
>
> Retaining address resolution unbundled with ESNI incentivizes the
> operator to use less number of keys; for example they might just use
> one key at a time. That would be a nice property to have, especially
> in terms of transparency. It would help third parties verify that an
> operator is actually providing an anonymity set (rather than just
> pretending to use ESNI).

I don't quite agree with this. Operators can already use different
keys across records if they so choose. That said, given the concern,
we could add advisory text suggesting that operators use as few ESNI
keys as possible to cover addresses.

> Also, I would like to point out that it is only when one key is used
> by each of the provider at a given time that it would be possible to
> use ESNI from a network that blocks DoH or DoT; a user can write the
> IP addresses of services he/she wants to access in the hosts file, and
> supply the ESNI key to the client he/she uses. Such a hack becomes
> difficult under the unified approach, where operators could synthesize
> different keys based on various conditions.

Good point!

>
> At the least, if we are to adopt #136, I think we should change the
> definition of record digest included in the ClientHello extension so
> that it does not cover the "address_set" extension of the ESNI record.
> Otherwise, we would be exposing to the wire the fingerprint of the set
> of IP addresses contained in the DNS response the client received
> through a secure channel, whereas in the split record approach you
> only leak the ESNI key being used and the IP address the client has
> chosen.

I'm not sure this is an issue. Clients that use HappyEyeballs will
already reveal the set of addresses by attempting to connect to them.

> As much as I like the idea of bundling IP addresses and ESNI key, I am
> concerned if the requirement to bundle different types of records is a
> requirement specific to ESNI. If not, I think we should (in the long
> term) try to create a generic solution, while in the short time we
> could rely on a more limited mechanism as proposed in #137.

In my view, since these are both extensions, they can both proceed in
parallel. If some operators can support #136, and can do so without
creating many ESNI keys, then I think it's worth including. Operators
for which this is not a concern do not need to support the extension,
right?

> > #137 builds on this design with a mechanism that lets
> > clients detect and recover from A/AAAA and ESNI mismatch (if desired).
> > It is certainly more complex in several respects. A third variant,
> > which is not (yet) in PR form, is a simplification of #137 wherein
> > ESNIKeys addresses are only used as filters, instead of filters *or*
> > complete addresses.
>
> I think using IP address blocks to limit the validity of the ESNI key
> is a fine approach. I do not oppose to having a name-based filter if
> it's impossible for certain operators to apply the IP address-based
> filtering approach.

Indeed -- that's the only reason the name exists.

Best,
Chris

---

From: Mike Bishop <[email-masked]>
Date: Fri, 1 Mar 2019 23:19:24 +0000

Stephen, there are a couple complicating factors here where I think we all have varying knowledge gaps.

  *   There are two major ways of pointing to a CDN:  Direct A/AAAA records and CNAMEs.  The easiest way to handle key update complexities on the part of the CDN(s) is simply to CNAME the ESNI query for your domain to their ESNI record, but you can certainly directly host your CDN’s keys in your domain if you prefer.  Nothing should preclude that.
  *   The issue we’re trying to address is when the ESNI record and the A/AAAA record follow different CNAME chains and you get the records from different hosts.  Of course, if we move to an ESNI RRType with the same hostname (see #105), there’s hopefully a single CNAME chain that gets cached at the resolver and used when looking up both queries.  If they remain separate hostnames, it seems like it gets much easier for them to diverge.
  *   It’s my understanding that DNSsec doesn’t play well with returning a subset of all extant RRs for a given name+type.  However, some CDNs return DNS results tailored to the user’s location; the load-balancing servers will (in some cases) return CNAMEs to different targets based on the desired traffic share.  That’s not a behavior that maps well to DNSsec as I understand it.  You mention DNSsec signing your domain as part of why you have issues with the proposal, but I think this is an open issue beyond ESNI or these PRs.

Maybe someone better-steeped in DNS/DNSsec can help us figure out how all that should work, and I agree with you that there are definitely bumps here we need to iron out – maybe those are just questions to answer, or maybe changes to the structure of the record are warranted.

However, these PRs are primarily about what information should be in these records and how clients make use of it.  I disagree with you that we have to resolve both questions at the same time.  Let’s agree on content first, and spent some time separately with DNS folks to see whether the content can be more pleasantly arranged.

-----Original Message-----
From: TLS <[email-masked]> On Behalf Of Stephen Farrell
Sent: Thursday, February 28, 2019 2:50 AM
To: Eric Rescorla <[email-masked]>
Cc: <[email-masked]> <[email-masked]>
Subject: Re: [TLS] Two Multi-CDN proposals

Hiya,

On 28/02/2019 02:40, Eric Rescorla wrote:

> On Wed, Feb 27, 2019 at 5:56 PM Stephen Farrell

> <[email-masked]<mailto:[email-masked]>>

> wrote:

>> Hiya,

>> On 28/02/2019 01:41, Eric Rescorla wrote:

>>> I think you're misunderstanding the scenario here: we have two CDNs

>>> A and B, and some switching service in front, so that when you

>>> lookup

>> example.com,

>>> you get a CNAME to A or B, and then you get the ESNIKeySet

>> (ESNIKeySet is a type you've just invented I guess?)

> No. I forgot it was named ESNIKeys

Phew:-)

>>> for A or B as

>>> the case may be. So you're not going to get both ESNIKeys values.

>> Yes, that's not the model I had in mind. I don't recall that having

>> been written down but maybe I missed it. (Where should I look?)

> I believe this was discussed in Bangkok during the discussion of

> problems with the current structure.

FWIW, I didn't take from that discussion that we only want that model to be supported, but it may have passed me by if that was stated.

>> The model I had in mind was where the hidden site has it's own DNS

>> operator but >1 CDN/front-site with each of those having a private

>> ESNI value. (And if there's some front-end DNS cleverness, it'd kick

>> in when the CNAME from #137 is being chased down.)

> I don't see how this is conflicts with what I said above, as that

> server still needs to ensure consistency.

I don't think mine conflicts with the model you describe, but I do think it has different consequences for how we ought structure the ESNIKeys stuff.

To be more specific, say in my model I have example.com and want to see ESNI used for www.example.com<http://www.example.com> and I publish the zone for example.com including ESNIKeys.

Now I want browsers to be able to use either cdn1.example or cdn2.example to front for www.example.com<http://www.example.com> where those are independent CDNs.

So I need to update my DNS zone periodically whenever one or other CDN changes their ESNI public key share.

In my tiny little case doing this for a few domains, I already have a small infrastructure that allows me to do that kind of thing because of the need for regular DNSSEC re-signing. (Mine currently works at a weekly or daily cadence, but doing it hourly would be fine.)

I'd like to do that via a simple update to my zone files without having to unpack and re-pack the data structures I get from cdn1 or cdn2. Now sure, I could write a new tool to munge together what I get from the CDNs but that's more work (that could go wrong) and doesn't match my current work flows. And I suspect others may operate similarly.

That's what leads me to think that we'd be better off to have multi-valued answers when a browser looks up the RRset at _esni.www.example.com with each separate value matching one ESNI public share (or one CDN, though I'd argue for one share per zone file stanza).

I don't think that conflicts with your model where _esni.www.example.com is one or another CNAME at a given moment but it does differ from it.

There is however some dependency on #137 to get what I want I guess using the host_pointer to get the privacy benefit of using a CDN. I guess I might need to publish yet another ESNI public share that matches the private available at the A/AAAA of www.example.com<http://www.example.com> as well as those from the CDN even though that may get me less privacy benefit compared to browsers who go to cdn1 or cdn2. (It's possible that I'm reading

#137 wrong though, but I read it as supporting the kind of setup I describe here.)

> In any case, the model I am describing has a consistency problem which

> needs to be addressed.

>> PS: I nonetheless maintain my points about the current ESNIKeys

>> structure - it's over generic and over complex and these PRs can only

>> make that worse:-)

> Yes, I am aware this is your opinion, but I don't agree.

Fair enough:-) Personally I think that if we support the kind of model above, such simplifications may well naturally fall out of that work but we'll see I guess.

For example, I think that'd allow re-structuring the ESNIKeys thing so the host_pointer in #137 no longer needs to be an extension and hence we don't need the concept of mandatory/critical extensions at all.

Cheers,

S.

> -Ekr

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 1 Mar 2019 23:52:32 +0000

Hiya,

> we all have varying knowledge gaps.
Doubtless. I confess lots of ignorance as to how CDNs operate.

> 
> *   There are two major ways of pointing to a CDN:  Direct A/AAAA
> records and CNAMEs.  The easiest way to handle key update
> complexities on the part of the CDN(s) is simply to CNAME the ESNI
> query for your domain to their ESNI record, but you can certainly
> directly host your CDN’s keys in your domain if you prefer.  Nothing
> should preclude that.`

I agree that we need something that works for both. As there
are meta-issues with some privacy mechanisms encouraging more
centralisation, I think it's important that we be explicit in
considering such, as we've been doing here. (To simplify and
paraphrase it a bit: part of my position is that "nothing
should preclude" is too weak - ESNI needs to work for both - to
the extent possible.)

> *   The issue we’re trying to address is when
> the ESNI record and the A/AAAA record follow different CNAME chains
> and you get the records from different hosts.  Of course, if we move
> to an ESNI RRType with the same hostname (see #105), there’s
> hopefully a single CNAME chain that gets cached at the resolver and
> used when looking up both queries.  If they remain separate
> hostnames, it seems like it gets much easier for them to diverge. 

I agree with all you say except the first one of the two words "the
issue" - there is more than one issue at play I'd say:-)

> *
> It’s my understanding that DNSsec doesn’t play well with returning a
> subset of all extant RRs for a given name+type.  However, some CDNs
> return DNS results tailored to the user’s location; the
> load-balancing servers will (in some cases) return CNAMEs to
> different targets based on the desired traffic share.  That’s not a
> behavior that maps well to DNSsec as I understand it.  You mention
> DNSsec signing your domain as part of why you have issues with the
> proposal, but I think this is an open issue beyond ESNI or these
> PRs.

It seems I didn't explain myself so well. For my setup the fact
that I support DNSSEC (and hence periodic re-signing) makes it
easier for me to support periodically acquiring and (re-)publishing
ESNIKeys from multiple sources. So having an existing DNSSEC setup
helps here.

Other than that, I'm not sure DNSSEC and ESNI have so much to do
with one another, given that DNSSEC really doesn't provide any
cryptographic linkage when things like MX or CNAME are used in the
DNS.

> Maybe someone better-steeped in DNS/DNSsec can help us figure out how
> all that should work, and I agree with you that there are definitely
> bumps here we need to iron out – maybe those are just questions to
> answer, or maybe changes to the structure of the record are
> warranted.
> 
> 
> 
> However, these PRs are primarily about what information should be in
> these records and how clients make use of it.  I disagree with you
> that we have to resolve both questions at the same time.  Let’s agree
> on content first, and spent some time separately with DNS folks to
> see whether the content can be more pleasantly arranged.

I'm not quite sure what you mean but I can think of two things:-

1. I have argued about how the content and structure of ESNIKeys
can affect deployment models and manageability, and do think that
now is the time to consider both. I maintain that position. And
my main points in this thread have all been about how the structure
of ESNIKeys could affect deployment models.

2. I think I explicitly said I'm fine with the encoding (TXT vs new
RRTYPE) being handled later. The same goes for a lot of other aspects
of the structure of ESNIKeys (but not all). There were a bunch of
mails though, so it's likely easy to miss that;-)

Cheers,
S.

> 
> 
> 
> -----Original Message----- From: TLS <[email-masked]> On Behalf
> Of Stephen Farrell Sent: Thursday, February 28, 2019 2:50 AM To: Eric
> Rescorla <[email-masked]> Cc: <[email-masked]> <[email-masked]> Subject:
> Re: [TLS] Two Multi-CDN proposals
> 
> 
> 
> 
> 
> Hiya,
> 
> 
> 
> FWIW, I didn't take from that discussion that we only want that model
> to be supported, but it may have passed me by if that was stated.
> 
> 
> 
> I don't think mine conflicts with the model you describe, but I do
> think it has different consequences for how we ought structure the
> ESNIKeys stuff.
> 
> 
> 
> To be more specific, say in my model I have example.com and want to
> see ESNI used for www.example.com<http://www.example.com> and I
> publish the zone for example.com including ESNIKeys.
> 
> 
> 
> Now I want browsers to be able to use either cdn1.example or
> cdn2.example to front for www.example.com<http://www.example.com>
> where those are independent CDNs.
> 
> 
> 
> In my tiny little case doing this for a few domains, I already have a
> small infrastructure that allows me to do that kind of thing because
> of the need for regular DNSSEC re-signing. (Mine currently works at a
> weekly or daily cadence, but doing it hourly would be fine.)
> 
> 
> 
> I'd like to do that via a simple update to my zone files without
> having to unpack and re-pack the data structures I get from cdn1 or
> cdn2. Now sure, I could write a new tool to munge together what I get
> from the CDNs but that's more work (that could go wrong) and doesn't
> match my current work flows. And I suspect others may operate
> similarly.
> 
> 
> 
> That's what leads me to think that we'd be better off to have
> multi-valued answers when a browser looks up the RRset at
> _esni.www.example.com with each separate value matching one ESNI
> public share (or one CDN, though I'd argue for one share per zone
> file stanza).
> 
> 
> 
> I don't think that conflicts with your model where
> _esni.www.example.com is one or another CNAME at a given moment but
> it does differ from it.
> 
> 
> 
> There is however some dependency on #137 to get what I want I guess
> using the host_pointer to get the privacy benefit of using a CDN. I
> guess I might need to publish yet another ESNI public share that
> matches the private available at the A/AAAA of
> www.example.com<http://www.example.com> as well as those from the CDN
> even though that may get me less privacy benefit compared to browsers
> who go to cdn1 or cdn2. (It's possible that I'm reading
> 
> Fair enough:-) Personally I think that if we support the kind of
> model above, such simplifications may well naturally fall out of that
> work but we'll see I guess.
> 
> For example, I think that'd allow re-structuring the ESNIKeys thing
> so the host_pointer in #137 no longer needs to be an extension and
> hence we don't need the concept of mandatory/critical extensions at
> all.
> 
> 
> 
> Cheers,
> 
> S.
> 
> 
> 
> 
> 
> 
> 
>> 
> 
>> -Ekr
> 
>>

---

From: Mike Bishop <[email-masked]>
Date: Sat, 2 Mar 2019 01:19:01 +0000

"The issue" the PRs are attempting to solve, not in terms of ESNI all-inclusive.  I think we're on the same page, then, except that I'm confused by two things in your reply that appear to be contradictory:  You want to address the content "and structure" of ESNIKeys now, but in the next paragraph reiterate that you're fine leaving what I see as the structural parts for later.

Reading through the thread again, I think I see at least one concrete issue with the current PRs that you raised:  The PRs currently assumes that they get back a single ESNI record and one or more A/AAAA records.  Depending on the deployment, it's possible that there are multiple ESNIKeys records and the draft explicitly permits this, but PR#137 currently does not describe the handling for that.  (PR#136 doesn't really need to, since it disregards the A/AAAA records.)

However, the more common deployment scenario for multi-CDN would be a single record (per version, eventually) from each CDN; each client would receive only one.

-----Original Message-----
From: Stephen Farrell <[email-masked]> 
Sent: Friday, March 1, 2019 3:53 PM
To: Mike Bishop <[email-masked]>; Eric Rescorla <[email-masked]>
Cc: <[email-masked]> <[email-masked]>
Subject: Re: [TLS] Two Multi-CDN proposals

Hiya,

> records and CNAMEs.  The easiest way to handle key update complexities 
> on the part of the CDN(s) is simply to CNAME the ESNI query for your 
> domain to their ESNI record, but you can certainly directly host your 
> CDN’s keys in your domain if you prefer.  Nothing should preclude 
> that.`

I agree that we need something that works for both. As there are meta-issues with some privacy mechanisms encouraging more centralisation, I think it's important that we be explicit in considering such, as we've been doing here. (To simplify and paraphrase it a bit: part of my position is that "nothing should preclude" is too weak - ESNI needs to work for both - to the extent possible.)

> to an ESNI RRType with the same hostname (see #105), there’s hopefully 
> a single CNAME chain that gets cached at the resolver and used when 
> looking up both queries.  If they remain separate hostnames, it seems 
> like it gets much easier for them to diverge.

I agree with all you say except the first one of the two words "the issue" - there is more than one issue at play I'd say:-)

> return DNS results tailored to the user’s location; the load-balancing 
> servers will (in some cases) return CNAMEs to different targets based 
> on the desired traffic share.  That’s not a behavior that maps well to 
> DNSsec as I understand it.  You mention DNSsec signing your domain as 
> part of why you have issues with the proposal, but I think this is an 
> open issue beyond ESNI or these PRs.

It seems I didn't explain myself so well. For my setup the fact that I support DNSSEC (and hence periodic re-signing) makes it easier for me to support periodically acquiring and (re-)publishing ESNIKeys from multiple sources. So having an existing DNSSEC setup helps here.

Other than that, I'm not sure DNSSEC and ESNI have so much to do with one another, given that DNSSEC really doesn't provide any cryptographic linkage when things like MX or CNAME are used in the DNS.

> answer, or maybe changes to the structure of the record are warranted.
> 
> 
> 
> on content first, and spent some time separately with DNS folks to see 
> whether the content can be more pleasantly arranged.

1. I have argued about how the content and structure of ESNIKeys can affect deployment models and manageability, and do think that now is the time to consider both. I maintain that position. And my main points in this thread have all been about how the structure of ESNIKeys could affect deployment models.

2. I think I explicitly said I'm fine with the encoding (TXT vs new
RRTYPE) being handled later. The same goes for a lot of other aspects of the structure of ESNIKeys (but not all). There were a bunch of mails though, so it's likely easy to miss that;-)

Cheers,
S.

---

From: Christopher Wood <[email-masked]>
Date: Fri, 1 Mar 2019 18:26:52 -0800

On Fri, Mar 1, 2019 at 3:19 PM Mike Bishop <[email-masked]> wrote:
> There are two major ways of pointing to a CDN:  Direct A/AAAA records and CNAMEs.  The easiest way to handle key update complexities on the part of the CDN(s) is simply to CNAME the ESNI query for your domain to their ESNI record, but you can certainly directly host your CDN’s keys in your domain if you prefer.  Nothing should preclude that.
> The issue we’re trying to address is when the ESNI record and the A/AAAA record follow different CNAME chains and you get the records from different hosts.  Of course, if we move to an ESNI RRType with the same hostname (see #105), there’s hopefully a single CNAME chain that gets cached at the resolver and used when looking up both queries.  If they remain separate hostnames, it seems like it gets much easier for them to diverge.
> It’s my understanding that DNSsec doesn’t play well with returning a subset of all extant RRs for a given name+type.  However, some CDNs return DNS results tailored to the user’s location; the load-balancing servers will (in some cases) return CNAMEs to different targets based on the desired traffic share.  That’s not a behavior that maps well to DNSsec as I understand it.  You mention DNSsec signing your domain as part of why you have issues with the proposal, but I think this is an open issue beyond ESNI or these PRs.
>

Thanks all for the discussion so far! Focusing strictly on the content
of the records and not the formatting, as Mike suggests, we
essentially have the following:

- #137 gives clients a way to detect A/AAAA+ESNI mismatch and recover,
at the cost of another sequential DNS query for ESNI. In this change,
A/AAAA records still control where traffic goes.
- #136 never requires clients to send a second DNS query for ESNI
since clients ignore the A/AAAA results. In this change, ESNI records
dictate routing.

With #137, clients willing to send a second DNS query will get ESNI
for all supporting providers. Clients unwilling to send a second DNS
query will only get ESNI for those providers which ensure that their
A/AAAA and ESNI records very rarely mismatch. With #136, clients only
get ESNI for those providers capable of building ESNI records with
correct addresses. In theory, these providers should be the same ones
that could ensure A/AAAA and ESNI record matching.

Given this, the discussion seems to hinge on the following question:
Are operators comfortable with the risks of letting ESNI records
control routing. If so, #136 is probably a better design for said
operators. If not, then #137 is probably required.

Note that this does not mean we must choose between #136 and #137
right now. We can do both (after possibly simplifying #137!), use
them, and see what works best in practice.

Anyway, I hope this summary accurately captures the differences and
possible tradeoffs.

Best,
Chris

---

From: Nick Sullivan <[email-masked]>
Date: Fri, 1 Mar 2019 18:39:04 -0800

On Fri, Mar 1, 2019 at 6:27 PM Christopher Wood <[email-masked]>
wrote:

> > There are two major ways of pointing to a CDN:  Direct A/AAAA records
> and CNAMEs.  The easiest way to handle key update complexities on the part
> of the CDN(s) is simply to CNAME the ESNI query for your domain to their
> ESNI record, but you can certainly directly host your CDN’s keys in your
> domain if you prefer.  Nothing should preclude that.
> > The issue we’re trying to address is when the ESNI record and the A/AAAA
> record follow different CNAME chains and you get the records from different
> hosts.  Of course, if we move to an ESNI RRType with the same hostname (see
> #105), there’s hopefully a single CNAME chain that gets cached at the
> resolver and used when looking up both queries.  If they remain separate
> subset of all extant RRs for a given name+type.  However, some CDNs return
> DNS results tailored to the user’s location; the load-balancing servers
> will (in some cases) return CNAMEs to different targets based on the
> desired traffic share.  That’s not a behavior that maps well to DNSsec as I
> understand it.  You mention DNSsec signing your domain as part of why you
> have issues with the proposal, but I think this is an open issue beyond
> ESNI or these PRs.
> >
> all that should work, and I agree with you that there are definitely bumps
> here we need to iron out – maybe those are just questions to answer, or
> maybe changes to the structure of the record are warranted.
> >
> these records and how clients make use of it.  I disagree with you that we
> have to resolve both questions at the same time.  Let’s agree on content
> first, and spent some time separately with DNS folks to see whether the
> content can be more pleasantly arranged.
>

Thanks for the summary, Chris.

Speaking for Cloudflare, we prefer the method described in #136 and would
be willing to implement ESNI records this way.

I have sympathy for organizations with a preference for #137 for
debuggability reasons, but I would rather avoid situations in which the
client needs to do an additional DNS query if avoidable.

I would support the option to include either extension based on operator
preference.

> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 1 Mar 2019 19:19:06 -0800

On Fri, Mar 1, 2019 at 6:39 PM Nick Sullivan <nick=
[email-masked]> wrote:

>
>
> On Fri, Mar 1, 2019 at 6:27 PM Christopher Wood <
> [email-masked]> wrote:
>
>> > The issue we’re trying to address is when the ESNI record and the
>> A/AAAA record follow different CNAME chains and you get the records from
>> different hosts.  Of course, if we move to an ESNI RRType with the same
>> hostname (see #105), there’s hopefully a single CNAME chain that gets
>> cached at the resolver and used when looking up both queries.  If they
>> remain separate hostnames, it seems like it gets much easier for them to
>> diverge.

This more or less aligns with my views.

>From my perspective, we know that #136 is safe, although it may be
inconvenient for some operators, and it's not clear to me that #137 can be
made to work without frequently incurring another serialized DNS query. If
we had some evidence to the contrary, I would be somewhat more favorable to
#137, though I would probably still prefer #136 for reasons of simplicity,
especially as we can always add #137 later if it proves viable.

-Ekr

---

From: Mike Bishop <[email-masked]>
Date: Sat, 2 Mar 2019 07:03:22 +0000

Totally agree that we want to avoid the extra DNS round-trip as often as possible.  However, I see the options in the opposite light – if all you need is #136, then you can put exact addresses into #137 and get the same behavior.  The question is whether the additional capabilities of #137 are safe and needed.  Depending how much later #137 is added, we may wind up needing to support both, which would be… suboptimal.

I think what will swing the needle on safety – how often there’s divergence – lies in how the record is positioned.  Two problems with the current approach that I see:

  *   Correct me if I’m wrong, but I don’t believe the alias is allowed to have subdomains.  If www.example.com<http://www.example.com> is a CNAME, then _esni.www.example.com cannot exist, can it?
  *   Even presuming that it did, or that it weren’t a subdomain, it would follow its own CNAME chain which might not lead to the correct provider.

If, however, the ESNIKeys RRType is resolved by following the CNAME from the host, it depends on how often two queries for two different RRTypes on the same hostname follow different CNAME chains.  We have a ready-made way to test that – A and AAAA.  I have an idea where we can get some data on that.

From: TLS <[email-masked]> On Behalf Of Eric Rescorla
Sent: Friday, March 1, 2019 7:19 PM
To: Nick Sullivan <nick=[email-masked]>
Cc: <[email-masked]> <[email-masked]>
Subject: Re: [TLS] Two Multi-CDN proposals

On Fri, Mar 1, 2019 at 6:39 PM Nick Sullivan <nick=[email-masked]<mailto:[email-masked]>> wrote:

On Fri, Mar 1, 2019 at 6:27 PM Christopher Wood <[email-masked]<mailto:[email-masked]>> wrote:

Speaking for Cloudflare, we prefer the method described in #136 and would be willing to implement ESNI records this way.

I have sympathy for organizations with a preference for #137 for debuggability reasons, but I would rather avoid situations in which the client needs to do an additional DNS query if avoidable.

I would support the option to include either extension based on operator preference.

From my perspective, we know that #136 is safe, although it may be inconvenient for some operators, and it's not clear to me that #137 can be made to work without frequently incurring another serialized DNS query. If we had some evidence to the contrary, I would be somewhat more favorable to #137, though I would probably still prefer #136 for reasons of simplicity, especially as we can always add #137 later if it proves viable..

-Ekr

Best,
Chris

_______________________________________________
TLS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tls
_______________________________________________
TLS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tls

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 2 Mar 2019 12:56:37 -0800

On Fri, Mar 1, 2019 at 11:03 PM Mike Bishop <[email-masked]> wrote:

> Totally agree that we want to avoid the extra DNS round-trip as often as
> possible.  However, I see the options in the opposite light – if all you
> need is #136, then you can put exact addresses into #137 and get the same
> behavior.
>

Sure, but if the error rate is too high, then people will just not do ESNI
with the non-exact address version, so you've absorbed complexity for
nothing.

i'd also like to hear from CDNs about whether their address ranges are
really small enough to not make the list of ranges prohobitive.

The question is whether the additional capabilities of #137 are safe and
> needed.  Depending how much later #137 is added, we may wind up needing to
> support both, which would be… suboptimal.
>

I actually don't think that's so bad. The complexity of the union of 136
and 137 isn't actually much  more than the complexity of 137, especially if
the client just omits step 1 of the 137 algorithm (for exact addresses).

I think what will swing the needle on safety – how often there’s divergence
> – lies in how the record is positioned.  Two problems with the current
> approach that I see:
>
>    - Correct me if I’m wrong, but I don’t believe the alias is allowed to
>    have subdomains.  If www.example.com is a CNAME, then _
>    esni.www.example.com cannot exist, can it?
>    - Even presuming that it did, or that it weren’t a subdomain, it would
>    follow its own CNAME chain which might not lead to the correct provider.
>
>
My understanding is that there was consensus to remove _esni, just that we
didn't get around to it because of this issue.

>
> If, however, the ESNIKeys RRType is resolved by following the CNAME from
> the host, it depends on how often two queries for two different RRTypes on
> the same hostname follow different CNAME chains.  We have a ready-made way
> to test that – A and AAAA.  I have an idea where we can get some data on
> that.
>

Great. I would be interested in seeing that.

-Ekr

>
> *From:* TLS <[email-masked]> *On Behalf Of * Eric Rescorla
> *Sent:* Friday, March 1, 2019 7:19 PM
> *To:* Nick Sullivan <nick=[email-masked]>
> *Cc:* <[email-masked]> <[email-masked]>
> *Subject:* Re: [TLS] Two Multi-CDN proposals
>
>
>
>
>
> especially as we can always add #137 later if it proves viable..
>
>
>
> -Ekr
>
>
>
>
>
>

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 5 Mar 2019 12:17:22 +0900

> keys as possible to cover addresses.

Having a guidance sounds sufficient to me.

> already reveal the set of addresses by attempting to connect to them.

Using happy eyeballs does not always mean that the clients are
exposing all the addresses in the address_set. IIUC, a client does not
eyeball between multiple addresses belonging to the same address
family.

> for which this is not a concern do not need to support the extension,

I think the idea of using extensions is a nice approach to move
forward. I won't oppose to having an "option" to change the way the
address is being resolved, as long as it does not hinder the
standardization of the draft without the feature.

--
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 5 Mar 2019 12:24:07 +0900

2019年3月3日(日) 5:57 Eric Rescorla <[email-masked]>:
>> Totally agree that we want to avoid the extra DNS round-trip as often as possible.  However, I see the options in the opposite light – if all you need is #136, then you can put exact addresses into #137 and get the same behavior.
>
>
> Sure, but if the error rate is too high, then people will just not do ESNI with the non-exact address version, so you've absorbed complexity for nothing.
>
> i'd also like to hear from CDNs about whether their address ranges are really small enough to not make the list of ranges prohobitive.
>
>
>> The question is whether the additional capabilities of #137 are safe and needed.  Depending how much later #137 is added, we may wind up needing to support both, which would be… suboptimal.
>
>
> I actually don't think that's so bad. The complexity of the union of 136 and 137 isn't actually much  more than the complexity of 137, especially if the client just omits step 1 of the 137 algorithm (for exact addresses).
>
>> Correct me if I’m wrong, but I don’t believe the alias is allowed to have subdomains.  If www.example.com is a CNAME, then _esni.www.example.com cannot exist, can it?
>> Even presuming that it did, or that it weren’t a subdomain, it would follow its own CNAME chain which might not lead to the correct provider.
>
>
> My understanding is that there was consensus to remove _esni, just that we didn't get around to it because of this issue.
>
>>
>>

FWIW, I am also looking forward to seeing the data, because if the
chances of responses getting out-of-sync is very low (e.g. 0.1%), I
think there's chance we might agree without having neither of #136 or
#137.

We could just use the ESNI key synchronization mechanism that was
introduced in #124 by using IP address-based certificates. That'd have
2 RT penalty (or 1 RT when TCP fast open is used), but that might be
tolerable if the probability is low.

>> On Fri, Mar 1, 2019 at 6:39 PM Nick Sullivan <nick=[email-masked]> wrote:
>>
>>
>>
>>
>>
>> On Fri, Mar 1, 2019 at 6:27 PM Christopher Wood <[email-masked]> wrote:
>>

-- 
Kazuho Oku

---

From: "Christopher Wood" <[email-masked]>
Date: Sat, 09 Mar 2019 18:07:40 -0800

On 4 Mar 2019, at 19:24, Kazuho Oku wrote:

>>> Totally agree that we want to avoid the extra DNS round-trip as 
>>> often as possible.  However, I see the options in the opposite light 
>>> – if all you need is #136, then you can put exact addresses into 
>>> #137 and get the same behavior.
>>
>>
>> Sure, but if the error rate is too high, then people will just not do 
>> ESNI with the non-exact address version, so you've absorbed 
>> complexity for nothing.
>>
>> i'd also like to hear from CDNs about whether their address ranges 
>> are really small enough to not make the list of ranges prohobitive.
>>
>>
>>> The question is whether the additional capabilities of #137 are safe 
>>> and needed.  Depending how much later #137 is added, we may wind up 
>>> needing to support both, which would be… suboptimal.
>>
>>
>> I actually don't think that's so bad. The complexity of the union of 
>> 136 and 137 isn't actually much  more than the complexity of 137, 
>> especially if the client just omits step 1 of the 137 algorithm (for 
>> exact addresses).
>>
>>> I think what will swing the needle on safety – how often there’s 
>>> divergence – lies in how the record is positioned.  Two problems 
>>> with the current approach that I see:
>>>
>>> Correct me if I’m wrong, but I don’t believe the alias is 
>>> allowed to have subdomains.  If www.example.com is a CNAME, then 
>>> _esni.www.example.com cannot exist, can it?
>>> Even presuming that it did, or that it weren’t a subdomain, it 
>>> would follow its own CNAME chain which might not lead to the correct 
>>> provider.
>>
>>
>> My understanding is that there was consensus to remove _esni, just 
>> that we didn't get around to it because of this issue.
>>
>>>
>>>
>>> If, however, the ESNIKeys RRType is resolved by following the CNAME 
>>> from the host, it depends on how often two queries for two different 
>>> RRTypes on the same hostname follow different CNAME chains.  We have 
>>> a ready-made way to test that – A and AAAA.  I have an idea where 
>>> we can get some data on that.
>>
>>

Absent this data and objections to #136, I’ve merged the PR as a 
viable mitigation to the multi-CDN problem. We can continue iterating on 
#137 and work on collecting this critical data as needed in parallel.

Best,
Chris

---

From: Erik Nygren <[email-masked]>
Date: Fri, 29 Mar 2019 17:44:07 -0400

Following the discussion this week I realized some other major issues we'll
need to make sure we cover:

1) Handling proxies here is going to be tricky.  The CONNECT generally
needs to specify the hostname which needs to go to the server which has the
ESNI key for what gets sent in the TLS handshake.  IPs don't come into play
here at all.  The only thing I can think of for handling this is to pass
the canonical name to the CONNECT when using a proxy, and making sure that
the canonical name is specific to a CDN.   There may be some related issues
in non-proxy environments.

2) The extension model breaks down if not all CDNs send it as mandatory.
In the hallway, Chris suggested we could require at least one extension be
manditory in any ESNIKey record in DNS.  There could be a bunch of similar
corner cases.  This issue also applies to switching off of using ESNIKeys
(eg, if there had been no extension included).

3) Trusting A and AAAA records from the EDNSKeys is going to break
environments relying on /etc/resolv.conf for spoofing to staging or other
testing environments.  (Services and Support staff will likely be unhappy
as they do this all the time.)

> >> are really small enough to not make the list of ranges prohobitive.

At least for one CDN, there are tens to hundreds of possible A/AAAA records
that could be used in a given cluster, and then many thousands of
clusters.  Especially on IPv4 this space is not dense as some comes from
local provider space.  (The net result for each is far more IPv4 and IPv6
addresses than can be enumerated reasonably.)

Some additional minor issues we'll want to address or specify, regardless,
if we take this path:

* We'll want to make sure to specify that clients must round-robin or
permute the A and AAAA records included in the address list.  Typically
most recursive and/or stub resolvers handle this, but since it's all in one
RR and not an RRSET it will be on clients to do this properly.

* We may wish to provide guidance on how to handle A vs AAAA (eg, reference
RFC 8305).  One thing that clients may lose out on is support features
provided by the OS, such as those which sort results based on past
knowledge about RTT and the like.

I'm increasingly thinking that while we may wish to define a
general-purpose ESNIKEY record for use by generic applications, we may wish
to define application-protocol specific use-cases and bindings for some of
the most persnickety applications.  For example, an HTTP-specific "HTTPS"
record that combined ALTSVC, ESNIKEY, and "ANAME" style information may
solve a bunch of these issues together.  I've been talking to some folks
and am tempted to try writing up a draft on this.  (Mail might be another
case that will just want its own binding...)

       Erik

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 30 Mar 2019 15:30:55 +0000

Hiya,

On 29/03/2019 21:44, Erik Nygren wrote:
> 1) Handling proxies here is going to be tricky.  The CONNECTi generally

What do you mean by "canonical name"? (I wish we had a canonical set
of definitions for the names involved in ESNI! Maybe I'll try craft
some if nobody else has...)

> (eg, if there had been no extension included).

IMO there should be no extensions defined or needed for ESNIKeys - we
have the RRTYPE and internal version number which should be enough.

> as they do this all the time.)

That kind of thing and your points about the number of addresses
involved make me wonder if #136 is really a viable approach.

I'm generally not sure if the problem motivating #136 will turn
out to be as bad as feared, esp. if the same DoH/DoT session can
be used for the ESNIKeys and A/AAAA queries (though I'm also not
sure if browsers could easily ensure that.) I guess maybe more
testing may tell us more as DoH/DoT get better deployed.

S.