Subject: [TLS] Two Multi-CDN proposals

From: Christopher Wood <[email-masked]>
Date: Wed, 27 Feb 2019 08:18:28 -0800

Hi folks,

Below are two PRs that seek to address the multi-CDN issue discussed
on this list and in meetings:

   1. https://github.com/tlswg/draft-ietf-tls-esni/pull/136
   2. https://github.com/tlswg/draft-ietf-tls-esni/pull/137

#136 implements the combined or stapled record approach discussed
several times, most recently in [1]. It includes these via an ESNIKeys
extension. #137 builds on this design with a mechanism that lets
clients detect and recover from A/AAAA and ESNI mismatch (if desired).
It is certainly more complex in several respects. A third variant,
which is not (yet) in PR form, is a simplification of #137 wherein
ESNIKeys addresses are only used as filters, instead of filters *or*
complete addresses.

We are asking for feedback on these PRs, as we would like to merge one
of them for the next draft version. As #136 is simpler and permits
extensibility, that is the current preference.

Thanks in advance for your feedback.

Best,
Chris (no hat, on behalf of the authors)

[1] https://mailarchive.ietf.org/arch/msg/tls/WXrPgaIsIPItDw3IQthmJk9VRlw

---

From: Mike Bishop <[email-masked]>
Date: Thu, 28 Feb 2019 00:36:42 +0000

Despite the additional complexity of #137, I think it's probably the better approach (and I would be fine with the simplification, if that makes it more palatable).  Particularly when multi-CDN is used, there's a lot of logic involved in generating the "right" A/AAAA record in response to a request.  #136 essentially lets the ESNIKeys result override the A/AAAA, which means ESNIKeys needs to be equally tailored and duplicate all the existing logic for a new record type.  In #137, the ESNIKeys essentially contains enough information to check that the A/AAAA results came from the same entity as the ESNIKeys result but leaves the DNS complexity where it already exists.  (It has the option to override, and removing that would be the simplification.)

The tripping point of #137, however, is that it may need a recovery path (i.e. a second A/AAAA resolution) in case the records come from mismatched providers.  We don't currently have data on how often that would happen -- whether it's 10% or 0.001% of the time would make a big difference.

-----Original Message-----
From: TLS <[email-masked]> On Behalf Of Christopher Wood
Sent: Wednesday, February 27, 2019 8:18 AM
To: <[email-masked]> <[email-masked]>
Subject: [TLS] Two Multi-CDN proposals

Hi folks,

Below are two PRs that seek to address the multi-CDN issue discussed on this list and in meetings:

#136 implements the combined or stapled record approach discussed several times, most recently in [1]. It includes these via an ESNIKeys extension. #137 builds on this design with a mechanism that lets clients detect and recover from A/AAAA and ESNI mismatch (if desired).

It is certainly more complex in several respects. A third variant, which is not (yet) in PR form, is a simplification of #137 wherein ESNIKeys addresses are only used as filters, instead of filters *or* complete addresses.

We are asking for feedback on these PRs, as we would like to merge one of them for the next draft version. As #136 is simpler and permits extensibility, that is the current preference.

Best,

_______________________________________________

TLS mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/tls

---

From: Christopher Wood <[email-masked]>
Date: Wed, 27 Feb 2019 16:54:41 -0800

> The tripping point of #137, however, is that it may need a recovery path (i.e. a second A/AAAA resolution) in case the records come from mismatched providers.  We don't currently have data on how o…

Thanks, Mike! I agree with the technical descriptions of both approaches.

Since each design uses an extension, could we pursue them
simultaneously? I'm not sure we must necessarily choose one or the
other here. (Perhaps I should have emphasized that in my original
message. Oops.)

Best,
Chris

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 28 Feb 2019 01:24:14 +0000

Hiya,

First, I think both are wrong:-)

If there are really different ESNI private value holders,
then each of those should provide separate ESNIKeys RR value
instances and the set of all of those should be in the RRset
returned when the ESNIKeys are queried.

Requiring different private value holders to find some
entity to wrap up all their crap into one single TLS blob
is not, IMO, a good design.

That said, we can fix that later I guess, even though we
will have to fix it sometime. (Fixing it now would be better!)
So I regard these ESNIKeys syntax decisions as only holding
temporarily.

>    2. https://github.com/tlswg/draft-ietf-tls-esni/pull/137

I'm not sure of the details of #137 but I think I prefer
that one a bit. I suspect either of them will be a pain to
code given how we'll be mixing names and addresses.

#136 seems broken to me in that it gives the new AddressSet
invention priority over A/AAAA resolution. I think that'd
need much more justification as it essentially calls for
duplicating A/AAAA information over two administrative
domains which seems like a recipe for failure.

If #137 is adding mandatory extensions that are analogous
to X.509 extension criticality, then I think that's an
error. Despite good arguments, that has not worked out well
and this seems like the same mistake. (But as stated above,
I think all syntax here is temporary until we've made it
DNS admin friendly which it is not.)

My reason for preferring #137 is that (if I'm reading it
right, and I'm not sure!), it resolves a CNAME and then
compares the resulting A/AAAA acquired via normal CNAME
chasing to a mask from the ESNIKeys stuff. I think that
is a better approach than replicating A/AAAA values inside
a novel repetitive structure like ESNIKeys.

Lastly, we will need to fix the TLS-developer-friendly
but DNS-admin-unfriendly ESNIKeys structure, so why not
do that now too, as we're modifying it? IMO the way to do
that is for each ESNI private value holder's stuff to be
a separate RR value in the RRset (and so each is likely a
separate stanza in a zone file). I also think if we did
that we'd be able to simplify the ESNIKeys structure a
good bit all of which should lead to more robust and
simpler code. My current code for handling ESNIKeys is
very complex and long even though I've ignored the supposed
ways in which things can be extended - were I to have to
add real extension handling, it'd be even worse. Changing
to >1 RR value in the answer could make all this an awful
lot simpler and much better match the multi-CDN (or more
properly multi-private-value) scenarios.

So in summary:

- both are wrong:-)
- if I had to pick one to proceed with temporarily,
  I'd go with #137 despite it being unclear
- I'd rather we fix the ESNIKeys to be DNS friendly
  now too and not wait to have to inevitably fight
  that battle later (but fight I will:-)

Cheers,
S.

> [email-masked]

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 27 Feb 2019 17:41:56 -0800

On Wed, Feb 27, 2019 at 5:24 PM Stephen Farrell <[email-masked]>
wrote:

> instances

Yes, this is the idea

and the set of all of those should be in the RRset
> returned when the ESNIKeys are queried.

I think you're misunderstanding the scenario here: we have two CDNs A and
B, and some switching service in front, so that when you lookup example.com,
you get a CNAME to A or B, and then you get the ESNIKeySet for A or B as
the case may be. So you're not going to get both ESNIKeys values.

> is not, IMO, a good design.

That's not the way this works.

> domains which seems like a recipe for failure.

This is not correct for the reasons I indicated above. What happens is that
the terminal CDN is responsible for publishing a record with its keys and
its A/AAAA values. Indeed, the source of the problem we are trying to solve
is that you might in successive resolutions get the CNAME for A and B, thus
having a keys/A mismatch.

> but DNS-admin-unfriendly ESNIKeys structure,

[Citation needed]

IMO the way to do
> separate stanza in a zone file).

This seems to rest on the same misunderstanding of the scenario in question.

-Ekr

So in summary:
> > [1]
> https://mailarchive.ietf.org/arch/msg/tls/WXrPgaIsIPItDw3IQthmJk9VRlw
> >

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 28 Feb 2019 01:55:57 +0000

Hiya,

On 28/02/2019 01:41, Eric Rescorla wrote:
> you get a CNAME to A or B, and then you get the ESNIKeySet 

(ESNIKeySet is a type you've just invented I guess?)

> for A or B as

Yes, that's not the model I had in mind. I don't recall that having
been written down but maybe I missed it. (Where should I look?)

The model I had in mind was where the hidden site has it's own DNS
operator but >1 CDN/front-site with each of those having a private
ESNI value. (And if there's some front-end DNS cleverness, it'd
kick in when the CNAME from #137 is being chased down.)

Cheers,
S.

PS: I nonetheless maintain my points about the current ESNIKeys
structure - it's over generic and over complex and these PRs can
only make that worse:-)

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 27 Feb 2019 18:40:23 -0800

> (ESNIKeySet is a type you've just invented I guess?)

No. I forgot it was named ESNIKeys

> been written down but maybe I missed it. (Where should I look?)

I believe this was discussed in Bangkok during the discussion of problems
with the current structure.

> kick in when the CNAME from #137 is being chased down.)

I don't see how this is conflicts with what I said above, as that server
still needs to ensure consistency. In any case, the model I am describing
has a consistency problem which needs to be addressed.

> structure - it's over generic and over complex and these PRs can

Yes, I am aware this is your opinion, but I don't agree.

-Ekr

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 27 Feb 2019 23:34:39 -0800

Hi Chris,

Thank you for writing down the PRs describing possible designs that we
might adopt. I think it helps a lot in understanding the details and
making accurate comparisons.

My comments inline.

2019年2月27日(水) 8:19 Christopher Wood <[email-masked]>:
> extension.

Reading the PR, I think that there is a mild privacy concern. The
issue with the unified record proposal is that it incentivizes the
operators to create more ESNI records possibly using different keys,
which is against the design principle of ESNI: provide anonymity set
by using the least number of keys.

Retaining address resolution unbundled with ESNI incentivizes the
operator to use less number of keys; for example they might just use
one key at a time. That would be a nice property to have, especially
in terms of transparency. It would help third parties verify that an
operator is actually providing an anonymity set (rather than just
pretending to use ESNI).

Also, I would like to point out that it is only when one key is used
by each of the provider at a given time that it would be possible to
use ESNI from a network that blocks DoH or DoT; a user can write the
IP addresses of services he/she wants to access in the hosts file, and
supply the ESNI key to the client he/she uses. Such a hack becomes
difficult under the unified approach, where operators could synthesize
different keys based on various conditions.

At the least, if we are to adopt #136, I think we should change the
definition of record digest included in the ClientHello extension so
that it does not cover the "address_set" extension of the ESNI record.
Otherwise, we would be exposing to the wire the fingerprint of the set
of IP addresses contained in the DNS response the client received
through a secure channel, whereas in the split record approach you
only leak the ESNI key being used and the IP address the client has
chosen.

As much as I like the idea of bundling IP addresses and ESNI key, I am
concerned if the requirement to bundle different types of records is a
requirement specific to ESNI. If not, I think we should (in the long
term) try to create a generic solution, while in the short time we
could rely on a more limited mechanism as proposed in #137.

> #137 builds on this design with a mechanism that lets

I think using IP address blocks to limit the validity of the ESNI key
is a fine approach. I do not oppose to having a name-based filter if
it's impossible for certain operators to apply the IP address-based
filtering approach.

-- 
Kazuho Oku

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 28 Feb 2019 10:50:29 +0000

Hiya,

> No. I forgot it was named ESNIKeys

Phew:-)

> with the current structure.

FWIW, I didn't take from that discussion that we only want
that model to be supported, but it may have passed me by
if that was stated.

> still needs to ensure consistency. 

I don't think mine conflicts with the model you describe,
but I do think it has different consequences for how we
ought structure the ESNIKeys stuff.

To be more specific, say in my model I have example.com
and want to see ESNI used for www.example.com and I
publish the zone for example.com including ESNIKeys.

Now I want browsers to be able to use either cdn1.example
or cdn2.example to front for www.example.com where those
are independent CDNs.

So I need to update my DNS zone periodically whenever
one or other CDN changes their ESNI public key share.
In my tiny little case doing this for a few domains, I
already have a small infrastructure that allows me to
do that kind of thing because of the need for regular
DNSSEC re-signing. (Mine currently works at a weekly
or daily cadence, but doing it hourly would be fine.)

I'd like to do that via a simple update to my zone files
without having to unpack and re-pack the data structures
I get from cdn1 or cdn2. Now sure, I could write a new
tool to munge together what I get from the CDNs but
that's more work (that could go wrong) and doesn't match
my current work flows. And I suspect others may operate
similarly.

That's what leads me to think that we'd be better off
to have multi-valued answers when a browser looks up
the RRset at _esni.www.example.com with each separate
value matching one ESNI public share (or one CDN,
though I'd argue for one share per zone file stanza).

I don't think that conflicts with your model where
_esni.www.example.com is one or another CNAME at a
given moment but it does differ from it.

There is however some dependency on #137 to get what
I want I guess using the host_pointer to get the
privacy benefit of using a CDN. I guess I might need
to publish yet another ESNI public share that matches
the private available at the A/AAAA of www.example.com
as well as those from the CDN even though that may
get me less privacy benefit compared to browsers who
go to cdn1 or cdn2. (It's possible that I'm reading
#137 wrong though, but I read it as supporting the
kind of setup I describe here.)

> In any case, the model I am describing

> Yes, I am aware this is your opinion, but I don't agree.
Fair enough:-) Personally I think that if we support
the kind of model above, such simplifications may well
naturally fall out of that work but we'll see I guess.
For example, I think that'd allow re-structuring the
ESNIKeys thing so the host_pointer in #137 no longer
needs to be an extension and hence we don't need the
concept of mandatory/critical extensions at all.

Cheers,
S.

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 28 Feb 2019 05:12:37 -0800

> that model to be supported, but it may have passed me by

No, it's merely the most challenging.

> my current work flows. And I suspect others may operate

The vast majority of people have someone else manage their DNS, so I'm sure
some others do, but I doubt many.

> though I'd argue for one share per zone file stanza).

I don't really understand your point, because this *is* the expected
design.

> concept of mandatory/critical extensions at all.

Well, obviously if you bake all the functionality into the record from the
start, you won't need extensions, mandatory or otherwise. But I don't think
this piece of functionality is well enough understood to tdo that with.

-Ekr

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 28 Feb 2019 13:51:16 +0000

Hiya,

> I don't really understand your point, because this *is* the expected

Seems one of us is confused then, likely me:-)

Lemme try clarify it out this way:

$ dig +short txt _esni.cloudflare.com
"/wEXMtleACQAHQAgsKfk/Hy67PiJCb68AVrT
C5jXkYnJK5UKCHcF4BAojAwAAhMBAQQAAAAAX
HPm0AAAAABce8/QAAA="

That's one encoded ESNIKeys value accessed just now.

The ietf.org domain has two TXT RR values, (actual content
elided below for brevity) so I get:

$ dig +short txt ietf.org
"somegooglestuff"
"lotsofspfstuff"

What I'd like to see for my earlier example.com case would
be like:

$ dig +short txt _esni.www.example.com
"/wEXMtleACQAHQAgsKfk/Hy67PiJCb68AVrT
C5jXkYnJK5UKCHcF4BAojAwAAhMBAQQAAAAAX
HPm0AAAAABce8/QAAA="
"/wHHBBOoACQAHQAg4YSfjSyJPNr1z3F8KqzB
NBnMejim0mJZaPmria3XsicAAhMBAQQAAAAAW
9pQEAAAAABb4jkQAAA="

Where the first is supplied to me by cdn1.example and the
second is from cdn2.example.

And have browsers and other ESNI clients be able to handle
that, picking whatever they consider the best of the usable
options presented.

Are we still on the same page? If so, great!

(And assuming we are: I'd subsequently suggest we can
simplify ESNIKeys so that it just has one KeyShareEntry per
RR value and not a list of those, since multiples for one
CDN or many CDNs can be handled already as above. I think
making that change, and perhaps other simplifying changes,
at the same time as moving from TXT to a new RRTYPE would
be a fine thing, but there's no pressing need to make all
those changes for the -03 draft or in #137 though, I'd be
happy if we're heading there a little later.)

Cheers,
S.

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 28 Feb 2019 06:18:23 -0800

> that, picking whatever they consider the best of the usable

My understanding is that this is problematic for DNS reasons, namely that
you are supposed to concatenate the records, and that we definitely need a
way to go above 255 bytes. But I'm no DNS expert and if there's a way to
have both of these, then I have no problem with it.

> CDN or many CDNs can be handled already as above.

Yes, I believe you've made this suggestion before, and I opposed it for the
same reason I oppose it now. While technically possible, it requires you to
duplicate everything else in the entry, which is bad, and worse the more
other stuff you put in the record (for instance, the public name).

-Ekr

I think

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 28 Feb 2019 14:42:43 +0000

Hiya,

> have both of these, then I have no problem with it.

Cool.

And same here wrt DNS expertise, but we can find some of
that I expect:-)

Cheers,
S.