Subject: [TLS] Server validation of a second ClientHello

From: Martin Thomson <[email-masked]>
Date: Fri, 08 Feb 2019 14:31:18 +1100

TLS 1.3 is pretty firm about what you can change in a second ClientHello.  It lists a small set of allowed changes to extensions (cookie, PSK binder, key shares, early data, and padding).  For the rest it says that nothing can change.  So clearly a client is in the wrong if it changes, adds, or removes an extension that isn't on this list.

The first question is whether a server should be required to check that only permitted changes are made to a second ClientHello.  

The second question is - if the server is not required to check for consistency - which ClientHello the server uses.

The third question is - if we allow checking at all - what the consistency check looks like.

My position is that the check cannot be mandatory, but we should allow servers to check.  One feature that we put significant effort into supporting was the notion of a stateless HelloRetryRequest, which I believe to be incompatible with checking (or at best extraordinately inconvenient).

With that answer in mind, if the second ClientHello might differ, I assert that the server uses that.  Hubert observes that this behaviour is not specified at all, which is entirely true.  I think that using the first ClientHello would be nonsensical, because the server needs to use the second ClientHello for those extensions that can change, like key share.

If we require or allow checking - and I think that we should definitely allow a check - the details of the conditions under which a check might fail aren't completely obvious for new extensions.  A new extension might be defined that has special handling in HelloRetryRequest, so it might be reasonable to allow new extensions to be changed.

Conversely, we might assume that anything that changes is going to be present in the HelloRetryRequest.  Therefore, only extensions that the server understands and includes in HelloRetryRequest can change.  That's not true for the early data extension, which is removed after HelloRetryRequest without any requirement for a server indicating support, but we might assume that extensions in the core spec have special status in this regard.

The simplest thing here seems to be a prohibition on checking unknown extensions for consistency between ClientHello1 and ClientHello2.

A final observation about checking extensions: it's not clear that we require consistent extension ordering.  If the key share extension changes, can it be reordered?  Can unchanged extensions be reordered?  Clearly neither results in a semantic change, but if a server that is required to check, might need to aggressively compress information about the first ClientHello in order to comply and extension ordering is an obvious thing to erase to save space in the cookie.

---

From: Hubert Kario <[email-masked]>
Date: Fri, 08 Feb 2019 13:53:44 +0100

On Friday, 8 February 2019 04:31:18 CET Martin Thomson wrote:
> TLS 1.3 is pretty firm about what you can change in a second ClientHello. 
> It lists a small set of allowed changes to extensions (cookie, PSK binder,
> key shares, early data, and padding).  For the rest it says that nothing
> can change.  So clearly a client is in the wrong if it changes, adds, or
> removes an extension that isn't on this list.
> 
> The first question is whether a server should be required to check that only
> permitted changes are made to a second ClientHello.

given need for GREASE and two dozen flags across OpenSSL history for 
workarounds for bugs in different implementations, the lesson seems to be that 
the receiving side should be as strict as it can (within protocol confines) or 
we risk making the non-conforming behaviour common
 
> The second question is - if the server is not required to check for
> consistency - which ClientHello the server uses.
> 
> My position is that the check cannot be mandatory, but we should allow
> servers to check.  One feature that we put significant effort into
> supporting was the notion of a stateless HelloRetryRequest, which I believe
> to be incompatible with checking (or at best extraordinately inconvenient).

"extraordinarily inconvenient" is description of the process of implementing 
any crypto and cryptographic protocols properly...
 
> With that answer in mind, if the second ClientHello might differ, I assert
> that the server uses that.  Hubert observes that this behaviour is not
> specified at all, which is entirely true.  I think that using the first
> ClientHello would be nonsensical, because the server needs to use the
> second ClientHello for those extensions that can change, like key share.

And I say that when you have already parsed, verified and sanity-checked the 
first set of extensions, it is much saner to reuse that work and only update 
the values that can change and ignore all other extensions...

There are multiple ways to look at this, and given that the standard is very 
firm that the 2nd CH needs to be as similar to the first one as possible, it 
is thus silent on which takes precedence in case they differ in general, 
because they can't.
 
> If we require or allow checking - and I think that we should definitely
> allow a check - the details of the conditions under which a check might
> fail aren't completely obvious for new extensions.  A new extension might
> be defined that has special handling in HelloRetryRequest, so it might be
> reasonable to allow new extensions to be changed.

yes, and we already had discussion on this ML about this: if the extension 
needs to change (based on protocol definition), the server MUST include it in 
HRR to signal to client, "yes, I do know about it, you can update it" and 
client MUST NOT change it otherwise
 
> The simplest thing here seems to be a prohibition on checking unknown
> extensions for consistency between ClientHello1 and ClientHello2.

that's definitely too harsh

> A final observation about checking extensions: it's not clear that we
> require consistent extension ordering.

the reordering is not in the list of allowed changes listed by the RFC

> If the key share extension changes,
> can it be reordered?

no

> Can unchanged extensions be reordered? 

no

> Clearly
> neither results in a semantic change, but if a server that is required to
> check, might need to aggressively compress information about the first
> ClientHello in order to comply and extension ordering is an obvious thing
> to erase to save space in the cookie.

the cookie can be up to 2^16 bytes long, even if client sends all 50 
extensions and spaces them with unknown extensions between, that's at most 20 
bytes per extension = 1000 bytes total extra space needed in cookie (32 bytes 
and 1600 bytes if you want to be very conservative)

and that's the excessive case, in usual case the server will not have to deal 
with more than half a dozen unknown extensions
-- 
Regards,
Hubert Kario
Senior Quality Engineer, QE BaseOS Security team
Web: www.cz.redhat.com
Red Hat Czech s.r.o., Purkyňova 115, 612 00  Brno, Czech Republic

---

From: Martin Thomson <[email-masked]>
Date: Mon, 11 Feb 2019 10:43:39 +1100

> we risk making the non-conforming behaviour common

I take something different from that experience.  There is value in having some implementations do strict checks, certainly.  However, in this case, I don't believe that it is reasonable to perform these checks (more below).
  
> the values that can change and ignore all other extensions...

As I mentioned, NSS does that work, at least partially, and then throws all of that state away when it sends HelloRetryRequest.  It does that so that HelloRetryRequest can be truly stateless (an advantage for DTLS).  Doing that work all over is trivial, bundling the necessary state, not.

> client MUST NOT change it otherwise

I remember having that discussion, but I guess the only way it made it into the spec was as:

   -  Other modifications that may be allowed by an extension defined in
      the future and present in the HelloRetryRequest.

Which isn't particularly direct, but I guess that's enough.

> and 1600 bytes if you want to be very conservative)

Yeah, that's ridiculously large.  With quite a few extensions supported, and many more unknown to us, the only way we might realistically ensure that the ClientHello doesn't change is to save a hash snapshot at every boundary where the cookie extension might be inserted or where an extension might be changed.  SHA-2 has a fairly small state to capture, but that's still nearly unbounded state.  With an amplification factor of up to 8, meaning that it could be more efficient to send the client its entire ClientHello in the cookie.

---

From: Hubert Kario <[email-masked]>
Date: Tue, 12 Feb 2019 13:52:43 +0100

On Monday, 11 February 2019 00:43:39 CET Martin Thomson wrote:
> > extensions and spaces them with unknown extensions between, that's at most
> > 20 bytes per extension = 1000 bytes total extra space needed in cookie
> > (32 bytes and 1600 bytes if you want to be very conservative)
> 
> Yeah, that's ridiculously large.  With quite a few extensions supported, and
> many more unknown to us, the only way we might realistically ensure that
> the ClientHello doesn't change is to save a hash snapshot at every boundary
> where the cookie extension might be inserted or where an extension might be
> changed.  SHA-2 has a fairly small state to capture, but that's still
> nearly unbounded state.  With an amplification factor of up to 8, meaning
> that it could be more efficient to send the client its entire ClientHello
> in the cookie.

I definitely won't claim that it is easy or straight-forward to do, I do claim 
that it is possible. Yes, sometimes it may mean that sending the literal CH in 
cookie may be more bandwidth efficient.

And regarding the specific extension in question, to verify that it didn't 
change between Hello's it is only 2 bytes extra in cookie. If that is too much 
already, I don't think that stateless HRR is something we will see in 
implementations for years to come.

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 12 Feb 2019 16:31:41 -0800

I concur with what I take to be MT's position here:

1. The client is clearly prohibited from changing most elements of the CH
(except for listed exceptions).
2. It's reasonable to check for and fail the handshake on any spec
violation except those where checking is explicitly forbidden (e.g., Must
Be Zero but Must be Ignored)
3. Nothing in the spec requires the server to check for this condition.

-Ekr

On Tue, Feb 12, 2019 at 4:53 AM Hubert Kario <[email-masked]> wrote:

> Red Hat Czech s.r.o., Purkyňova 115, 612 00  Brno, Czech
> Republic_______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>

---

From: Hubert Kario <[email-masked]>
Date: Wed, 13 Feb 2019 13:12:34 +0100

> 3. Nothing in the spec requires the server to check for this condition.

and what about values that technically can't change (as you noted yourself) 
but they do change and the server does use them?

you are not suggesting that which value will be used (from first or second 
CH), or if the connection will be aborted, to be implementation dependant 
*by design* , do you?
 
> > > > extensions and spaces them with unknown extensions between, that's at
> > 
> > most
> > 
> > > Yeah, that's ridiculously large.  With quite a few extensions supported,
> > 
> > and
> > 
> > > the ClientHello doesn't change is to save a hash snapshot at every
> > 
> > boundary
> > 
> > > where the cookie extension might be inserted or where an extension might
> > 
> > be
> >

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Feb 2019 06:39:03 -0800

> CH), or if the connection will be aborted, to be implementation dependant

I'n not sure I understand your question, but I'll try to answer what I
think it says.

1. I do think that whether you continue the connection or abort it is an
implementation decision and I think that the way the spec is written says
that.
2. I think the spec leaves open whether you should use the first or second
values, but I think implementations should use the second value. It's not
clear why one would want to use the first.

To my knowledge, there was never a WG discussion about this exact question,
so we only have the spec to guide us. Were we pre-publication I would have
advocated for (a) leaving open whether to abort and (b) requiring you to
use the second value.

-Ekr

> > > I definitely won't claim that it is easy or straight-forward to do, I
> do
> > > claim
> > > that it is possible. Yes, sometimes it may mean that sending the
> literal
> > > CH in
> > > change between Hello's it is only 2 bytes extra in cookie. If that is
> too
> > > much

---

From: Ilari Liusvaara <[email-masked]>
Date: Wed, 13 Feb 2019 16:52:10 +0200

> clear why one would want to use the first.

One reason is that if one has state-machine handshake processing, it is
not difficult to do all the negotiation on the first pass, and then
only grab the share on second pass, if it was missing (as it is the
only retry condition).

That is what I initially implemented in TLS library I did (later
versions use stateless approach, with checks that the CHs match).

-Ilari

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 13 Feb 2019 09:04:54 -0600

> Which isn't particularly direct, but I guess that's enough.

I think I pressed for that text, on the grounds that the core spec should state the
core expectations but try to assume as little as possible about future extensions' needs.
It's pretty clear that this implies that (with the current set of extensions) a current
server can rely on the "only extensions in HRR can change" property (modulo the existing
exceptions in the core spec).

-Ben

---

From: Hubert Kario <[email-masked]>
Date: Wed, 13 Feb 2019 16:39:49 +0100

> clear why one would want to use the first.

because you have already parsed, verified and sanity-checked the first hello, 
you already decided what kind of parameters will the connection use and you're 
expecting just the values that can change to change and ignoring everything 
else, thus not wasting cycles on verifying the extensions twice...

so it's not clear to me why you'd ever want to use the second one

> advocated for (a) leaving open whether to abort and (b) requiring you to

so you don't think this qualifies for Errata?

side note: it is actually possible to check if the 2nd CH has minimal changes 
in stateless HRR by replacing the extensions that can change with the hashes 
of their values and then storing the overall hash of CH message together with 
hashes of the extensions that can change. When receiving the 2nd CH, we can 
replace the extensions that can change with respective hash values and then 
check if the overall hash still matches. That translates to something like 4 
hashes, two positions and two flags (for padding and early_data). Only this 
much is necessary to check perfectly if the CH is unchanged. The only 
exception is pre_shared_key, but server behaviour with it is clearly defined - 
the server must use the values from 2nd CH as the selected_identity wouldn't 
match otherwise and binders couldn't be verified. With addition of one more 
hash in the cookie the server may easily ensure that the identity it planned 
to select remained in the CH. That's like 165 bytes when using non-truncated 
sha-256 hashes.

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 13 Feb 2019 10:17:43 -0600

> so you don't think this qualifies for Errata?

Errata are used for things that were errors at the time of publication (and,
presumably, had WG consensus for being errors).  Since there was apparently no
WG consensus on this specific topic at time of publication, the errata process
seems inappropriate.

-Ben

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Feb 2019 08:31:52 -0800

On Wed, Feb 13, 2019 at 7:39 AM Hubert Kario <[email-masked]> wrote:

> > > > 1. The client is clearly prohibited from changing most elements of
> the
> > > > CH

As MT says, we use a stateless design, and we don't necessarily verify the
entire
CH prior to sending HRR. For instance, it's not necessary to do certificate
selection
and hence not to look at signature_algorithms.

> so it's not clear to me why you'd ever want to use the second one

Well, clearly views differ on this, then.

> so you don't think this qualifies for Errata?

What Ben said.

-Ekr

---

From: Hubert Kario <[email-masked]>
Date: Wed, 13 Feb 2019 19:23:42 +0100

> Well, clearly views differ on this, then.

yes, that was my point, and the reason why I'd like to see clarification or 
agreement on expected behaviour

so if my understanding is correct, to do that, we would need to agree on a new 
RFC that clarifies such issues
-- 
Regards,
Hubert Kario
Senior Quality Engineer, QE BaseOS Security team
Web: www.cz.redhat.com
Red Hat Czech s.r.o., Purkyňova 115, 612 00  Brno, Czech Republic

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Feb 2019 11:01:10 -0800

> RFC that clarifies such issues

I'm not so sure.

- I don't think that the specification ought to require checking and/or not
checking.
- I don't think it should require using the first CH, and unsure whether we
should require use of the second CH, but it doesn't sound like there's
consensus on that.

-Ekr

--

---

From: Hubert Kario <[email-masked]>
Date: Wed, 13 Feb 2019 20:37:41 +0100

On Wednesday, 13 February 2019 20:01:10 CET Eric Rescorla wrote:
> > > > > I'n not sure I understand your question, but I'll try to answer what
> > 
> > I
> > 
> > > > > think it says.
> > > > > 
> > > > > 1. I do think that whether you continue the connection or abort it
> > 
> > is an
> > 
> > > > > values, but I think implementations should use the second value.
> > > > > It's
> > > > > not
> > > > you already decided what kind of parameters will the connection use
> > > > and
> > > > you're

I said "I'd like", yes, I am aware that not everybody shares my views on those 
topics, but *if* we reach a conclusion and decide to update/clarify RFC 8446, 
it would require publishing a new RFC (depending on intrusiveness of it, it 
may be just "Informational" or it may need to be "Standards Track", with 
everything that it entails), wouldn't it?
 
> should require use of the second CH, but it doesn't sound like there's

it all depends on your threat model, if leaking information which kind of 
implementation you are running is useful information to the attacker, then 
having multiple implementations that are indistinguishable on protocol level 
may be very desirable

and in this specific example of record_size_limit, lack of agreement over 
extension from which Client Hello takes precedence will easily lead to 
interoperability failures

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Feb 2019 12:27:46 -0800

On Wed, Feb 13, 2019 at 11:37 AM Hubert Kario <[email-masked]> wrote:

> > > > > > implementation decision and I think that the way the spec is
> written
> > > > > > says
> > > > > > that.
> > > > > > 2. I think the spec leaves open whether you should use the first
> or
> > > > > > second
> > > > > because you have already parsed, verified and sanity-checked the
> first
> > > > > hello,
> > > yes, that was my point, and the reason why I'd like to see
> clarification
> > > or
> > > so if my understanding is correct, to do that, we would need to agree
> on a
> > > new

The current document does not mandate any particular implementation. If you
want to require a specific implementation then would need a new standards
track RFC.

> having multiple implementations that are indistinguishable on protocol

There are an enormous number of ways in which TLS stacks can vary and
requiring indistinguishable behavior is not a goal of the TLS spec.

> interoperability failures

You're forbidden to change that value, so you'll only see interop failures
if the client is already nonconformant.

-Ekr