Subject: Re: New Version Notification for draft-thomson-http-replay-00.txt

From: Subodh Iyengar <[email-masked]>
Date: Wed, 19 Jul 2017 14:20:14 +0000

Martin, a few others and I discussed this draft offline just after the HTTP WG meeting and I believe there is an extension of the dkg style attack possible on the proposal currently.

I'm making the following assumptions:
     * There is no special API to handle 0-rtt data by the TLS terminator, i.e. it is treated as a part of the same 1-rtt stream of data

  *   A Terminating proxy uses one of 2 approaches to decide to set Early-Data upstream:
     *   The terminating proxy just checks for isHandshakeFinished() to determine whether or not the data was send on 0-rtt or not, and sets the Early-Data accordingly
     *   The server buffers the entire request till it gets the Finished message and then forwards the requests without early data header
  *   There are 2 timeouts on the client, a transport timeout which is larger and a request timeout which is smaller
  *   The client will potentially retry requests that timeout potentially on a new connection
  *   A client decides to send a non-idempotent request over 0-rtt and relies on the server to reject it

Let's say the following things happen:

  *   A client sends a request to the Terminating proxy over 0-rtt.
  *   A MITM forwards the client hello to the proxy, gets the server hello etc. and forwards it to the client and gets the earlydata and finished message.
  *   The MITM then holds back the request and the finished message.
  *   The request would time out on the client and the client retries on a new connection over 1-rtt to another server.
  *   The transport has not timed out yet.
  *   The MITM then releases the original data to the proxy
  *   Since the proxy received the data and finished together, it would consider it to not be early data and not forward the early data header to the upstream
  *   The upstream server now has received the same request twice without knowing that it was sent over 0-rtt so it would not reject them and if it didn't have another idempotency mechanism, would execute it twice.

Any strategy that does not provide a custom API for servers to tell the difference between 0-rtt and non 0-rtt data suffers from this problem. However custom APIs are painful to work with.

Fortunately I think there is a simple fix to this though which is setting the early data header from the client directly and then the proxy can just forward it through. I understand that this cannot be exactly determined by the client, but it could be conservative about it.

Subodh
________________________________
From: Mike Bishop <[email-masked]>
Sent: Thursday, June 22, 2017 3:40:03 PM
To: Martin Thomson
Cc: HTTP Working Group
Subject: RE: New Version Notification for draft-thomson-http-replay-00.txt

Ah, just terminology, then.  I was reading "gateway" as "intermediary," not as "reverse proxy."  A reverse proxy is much more likely to have a close configuration relationship with its back-end server(s).

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]]
Sent: Thursday, June 22, 2017 3:24 PM
To: Mike Bishop <[email-masked]>
Cc: HTTP Working Group <[email-masked]>
Subject: Re: New Version Notification for draft-thomson-http-replay-00.txt

On 23 June 2017 at 03:38, Mike Bishop <[email-masked]> wrote:
> I like most of it, but the second paragraph in 5 seems a little hand-wavy.  The gateway is supposed to "know" the server supports this new standard, which it can only fully do if it has received a 4NN in the past, which would only happen if it knew in the past, which....  Chicken, meet egg.

I don't think that it is as hand-wavy as all that.  Keep in mind that as a gateway for HTTPS, the gateway has the keys for the origin server.  That means that there is a pretty strong relationship there.
The gateway can use that.  For instance, a CDN might delay requests unconditionally unless their customer has provided an override that enables immediate forwarding.

This is less of a chicken and egg issue even if the gateway is forced to hold requests.  As we well know, the number of requests that can fit into the first round trip is limited.  That's why header compression is so useful.  Early data gives the client a little more space for sending requests.  It's not much, but it's something.

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 19 Jul 2017 16:32:25 +0200

2017-07-19 16:20 GMT+02:00 Subodh Iyengar <[email-masked]>:
> Martin, a few others and I discussed this draft offline just after the HTTP
> WG meeting and I believe there is an extension of the dkg style attack
> possible on the proposal currently.
>
>      * There is no special API to handle 0-rtt data by the TLS terminator,
> i.e. it is treated as a part of the same 1-rtt stream of data
>
> A Terminating proxy uses one of 2 approaches to decide to set Early-Data
> upstream:
>
> The terminating proxy just checks for isHandshakeFinished() to determine
> whether or not the data was send on 0-rtt or not, and sets the Early-Data
> accordingly
> The server buffers the entire request till it gets the Finished message and
> then forwards the requests without early data header
>
> There are 2 timeouts on the client, a transport timeout which is larger and
> a request timeout which is smaller
> The client will potentially retry requests that timeout potentially on a new
> connection
> A client decides to send a non-idempotent request over 0-rtt and relies on
> the server to reject it
>
>
> A client sends a request to the Terminating proxy over 0-rtt.
> A MITM forwards the client hello to the proxy, gets the server hello etc.
> and forwards it to the client and gets the earlydata and finished message.
> The MITM then holds back the request and the finished message.
> The request would time out on the client and the client retries on a new
> connection over 1-rtt to another server.
> The transport has not timed out yet.
> The MITM then releases the original data to the proxy
> Since the proxy received the data and finished together, it would consider
> it to not be early data and not forward the early data header to the
> upstream
> The upstream server now has received the same request twice without knowing
> that it was sent over 0-rtt so it would not reject them and if it didn't
> have another idempotency mechanism, would execute it twice.

While I understand that such an issue exists, I am not sure if it is a
replay attack.

To me it seems like a retry attack.

It seems to me that the impact of the attack is equal to when a MITM
holds back the packet holding an HTTP response to cause the client
retry the request.

Or am I missing something?

> Any strategy that does not provide a custom API for servers to tell the
> difference between 0-rtt and non 0-rtt data suffers from this problem.
> However custom APIs are painful to work with.
>
> Fortunately I think there is a simple fix to this though which is setting
> the early data header from the client directly and then the proxy can just
> forward it through. I understand that this cannot be exactly determined by
> the client, but it could be conservative about it.
>
> Subodh
> Ah, just terminology, then.  I was reading "gateway" as "intermediary," not
> as "reverse proxy."  A reverse proxy is much more likely to have a close
> configuration relationship with its back-end server(s).
>
>> I like most of it, but the second paragraph in 5 seems a little hand-wavy.
>> The gateway is supposed to "know" the server supports this new standard,
>> which it can only fully do if it has received a 4NN in the past, which would
>> only happen if it knew in the past, which....  Chicken, meet egg.
>
> I don't think that it is as hand-wavy as all that.  Keep in mind that as a
> gateway for HTTPS, the gateway has the keys for the origin server.  That
> means that there is a pretty strong relationship there.
> The gateway can use that.  For instance, a CDN might delay requests
> unconditionally unless their customer has provided an override that enables
> immediate forwarding.
>
> This is less of a chicken and egg issue even if the gateway is forced to
> hold requests.  As we well know, the number of requests that can fit into
> the first round trip is limited.  That's why header compression is so
> useful.  Early data gives the client a little more space for sending
> requests.  It's not much, but it's something.

-- 
Kazuho Oku

---

From: Subodh Iyengar <[email-masked]>
Date: Wed, 19 Jul 2017 14:49:32 +0000

replay attack.

A better way to think about it might be that mItm could always hold back the request even now, but he can't confuse the origin that the request came from 0-rtt or not 0-rtt because no such promise exists right now, however with this mechanism he can confuse the backend with a retry. So I think such an issue should be in scope for this discussion.

I don't think this mechanism is meant to stop replays, it's just meant to give the app the right information for it to make a decision of whether or not the client followed it's policy of sending data in 0-rtt. Replays are protected more by the transport rather than the application. For example if I had a transport without replay protection, this mechanism would not protect me from the classic replay safety attack of an attacker sending "safe" requests again and again and getting information from the length of the repsonse, if I was okay with the request being sent over 0-rtt.

Subodh

________________________________
From: Kazuho Oku <[email-masked]>
Sent: Wednesday, July 19, 2017 7:32:25 AM
To: Subodh Iyengar
Cc: Mike Bishop; Martin Thomson; HTTP Working Group
Subject: Re: New Version Notification for draft-thomson-http-replay-00.txt

--
Kazuho Oku

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 19 Jul 2017 11:05:43 -0500

I agree with your analysis and that the proposed mitigation should have
the desired effect.

It's probably worth calling out that the scenario assumes the client
sends non-idempotent stuff over 0-RTT, which the client is not supposed
to do.  And, as you note, a server with a separate API/stream for early
data would know that the request in question came over early data.

-Ben

On 07/19/2017 09:20 AM, Subodh Iyengar wrote:
> Martin, a few others and I discussed this draft offline just after the
> HTTP WG meeting and I believe there is an extension of the dkg style
> attack possible on the proposal currently.
>
>      * There is no special API to handle 0-rtt data by the TLS
> terminator, i.e. it is treated as a part of the same 1-rtt stream of data
>
>       o The terminating proxy just checks for isHandshakeFinished() to
>         determine whether or not the data was send on 0-rtt or not,
>         and sets the Early-Data accordingly
>       o The server buffers the entire request till it gets the
>         Finished message and then forwards the requests without early
>         data header
>   * There are 2 timeouts on the client, a transport timeout which is
>     larger and a request timeout which is smaller
>   * Aclient sends a request to the Terminating proxy over 0-rtt.
>   * A MITM forwards the client hello to the proxy, gets the server
>     hello etc. and forwards it to the client and gets the earlydata
>     and finished message. 
>   * The request would time out on the client and the client retries on
>     a new connection over 1-rtt to another server. 
>   * Since the proxy received the data and finished together, it would
>     consider it to not be early data and not forward the early data
>     header to the upstream
>   * The upstream server now has received the same request twice
>     without knowing that it was sent over 0-rtt so it would not reject
>     them and if it didn't have another idempotency mechanism, would
>     execute it twice.
>
> Any strategy that does not provide a custom API for servers to tell
> the difference between 0-rtt and non 0-rtt data suffers from this
> problem. However custom APIs are painful to work with.
>
> Fortunately I think there is a simple fix to this though which is
> setting the early data header from the client directly and then the
> proxy can just forward it through. I understand that this cannot be
> exactly determined by the client, but it could be conservative about it.
>
> Subodh
> ------------------------------------------------------------------------
> *From:* Mike Bishop <[email-masked]>
> *Sent:* Thursday, June 22, 2017 3:40:03 PM
> *To:* Martin Thomson
> *Cc:* HTTP Working Group
> *Subject:* RE: New Version Notification for
> draft-thomson-http-replay-00.txt
>  
> Ah, just terminology, then.  I was reading "gateway" as
> "intermediary," not as "reverse proxy."  A reverse proxy is much more
> likely to have a close configuration relationship with its back-end
> server(s).
>
> > I like most of it, but the second paragraph in 5 seems a little
> hand-wavy.  The gateway is supposed to "know" the server supports this
> new standard, which it can only fully do if it has received a 4NN in
> the past, which would only happen if it knew in the past, which.... 
> Chicken, meet egg.
>
> I don't think that it is as hand-wavy as all that.  Keep in mind that
> as a gateway for HTTPS, the gateway has the keys for the origin
> server.  That means that there is a pretty strong relationship there.
> unconditionally unless their customer has provided an override that
> enables immediate forwarding.
>
> This is less of a chicken and egg issue even if the gateway is forced
> to hold requests.  As we well know, the number of requests that can
> fit into the first round trip is limited.  That's why header
> compression is so useful.  Early data gives the client a little more
> space for sending requests.  It's not much, but it's something.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 20 Jul 2017 06:25:24 +0200

Hi,

>      * There is no special API to handle 0-rtt data by the TLS terminator, i.e. it is treated as a part of the same 1-rtt stream of data

Does such an implementation really exist ? I mean, for openssl, it's clearly
different :

    https://www.openssl.org/docs/manmaster/man3/SSL_read_early_data.html

So you read early data using SSL_read_early_data() and this one returning
even one single byte indicates that you're getting early data. SSL_read()
will not work before the handshake completes.

I understand how having such an implementation could be an issue but then
this is probably something that should be documented in the draft, explaining
that one of the prerequisites to map HTTP on top of 0-RTT TLS, it's critical
that the API clearly distinguishes early data from regular data.

(...)
> However custom APIs are painful to work with.

One more reason for documenting this case clearly so that there is no
confusion about it!

> Fortunately I think there is a simple fix to this though which is setting the
> early data header from the client directly and then the proxy can just

Interestingly, we started with this initially, and figured that this was
completely covered by the distinction of early data from regular data,
and caused two other issues :
  - for some clients, changing a header field after a request was attempted
    and before operating a retry might be complicated (ie: the retry will
    very likely happen only at the transport layer)

  - whenever a proxy doesn't support early data, and would receive the whole
    request over regular data, it would have to strip this header field from
    the request. This results in having to deal with 3 values for this header
    field to know whether an attempt to use early data was made by the client,
    and whether at least one proxy in the chain received the request as early
    data.

While I agree that APIs are difficult, I tend to think that the purpose of
the draft is to explain the prerequisites to map HTTP on top of 0-RTT and
that instead of complexifying all implementations and making them more
fragile, we should rather indicate in which cases it's forbidden to enable
0-RTT (ie: SSL implementation doesn't provide a clear distinction between
early data and regular data). This could possibly be justified by using
your attack as an example.

Regards,
Willy

---

From: Martin Thomson <[email-masked]>
Date: Thu, 20 Jul 2017 09:59:39 +0200

> Does such an implementation really exist ? I mean, for openssl, it's clearly

There's a split in the community on this point.  NSS merges the
streams and 0-RTT reads and writes are not different from other reads
and writes.

> your attack as an example.

While I think that the example is relatively far-fetched, I think that
we should take the conservative approach here.  It means that the
client needs to strip the header field when writing.  That's a little
costly, but I think that we already have to deal with rebuilding
requests: if 0-RTT is rejected, then you can end up having to send an
h2 request with h1.  We also have this same requirement to change a
header field with token binding.

I'll write up a PR that makes the change: it actually simplifies the
spec a little.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 20 Jul 2017 10:22:49 +0200

> streams and 0-RTT reads and writes are not different from other reads

OK then I agree it's important to take care of this. One could say that
NSS transparently merging safe and unsafe data could cause trouble over
the long term and should possibly be adapted, but I don't know if that's
still possible.

> While I think that the example is relatively far-fetched,

Well, all security-related examples seem far-fetched until they become
exploitable vulnerabilities :-)

> I think that
> we should take the conservative approach here.

I agree on being conservative, which is also why I'm trying to check if
instead we could suggest not to implement it if the underlying layer
doesn't provide the necessary guarantees regarding received data. Do you
have contacts with the people working on NSS to know if that would be
something making sense for them, based on this example ?

> It means that the

Yes, that requires at least two distinct values as we initially had and
a slightly more complex processing along the chain. That's why I'm still
thinking that we could be more conservative by suggesting to only implement
if 100% safe.

> I'll write up a PR that makes the change: it actually simplifies the

I'm not completely sure, as we will rely on header field values and will
have to enforce correctness checks (and absence of duplication), which is
a bit more painful than simply checking for header field presence to notify
about a risk.

Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 20 Jul 2017 13:12:23 -0500

> something making sense for them, based on this example ?

The debate in the TLS WG over the question of needing separate APIs for
a (potentially) ideologically separate data stream was ... quite
heated.  So I was hoping to not reopen it here, given that we seem to
have ended up at a consensus that it is possible to reason correctly and
obtain correct results using either philosophical picture.

Perhaps there would only need to be a new API for "did any data on this
stream arrive as early data" (in addition to the "is the handshake
finished" API already known about).

-Ben

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 21 Jul 2017 09:07:55 +0200

Hi Ben,

> finished" API already known about).

I'm fine with not reopening the debate, but at least we need to know what
grounds we're basing our design on. I understood during discussions at the
workshop that if the server receives the final handshake, it's a guarantee
that the client is the one we expect. But your example shows that it's not
sufficient to distinguish between really safe data and possibly replayed
ones (though my understanding is that the request could only be replayed
once, on the originally established connection).

If we push the concept a bit further, the issue is more widely related to
retries in general, because the same principle could be used to force a
client to retry upon any timeout by holding on its data. Even a plain
regular HTTPS request (you just have to hold the last part containing
the request, delay it for a while and wait for the client to retry, or
even the user to press F5).

Thus what I'm seeing is that the root of the problem is not at all related
to the way we map HTTP on top of TLS, but to the fact that whenever there
is a possibility of retry, there can be a duplicate request. And in fact
it's a well known problem for those of us having to implement persistent
connections to servers.

The draft suggests that the client may only retry after seeing the 4NN
status code, it doesn't suggest it's allowed before. The intercepting
proxy trying to replay the request in your case will not be able to
deliver this 4NN status code as it would require to spoof the server's
connection. So in such a case there's no reason for the client to retry.

Probably that we should make it clearer in the draft that the client must
not retry on other situations ? But then I fear that we'd be walking on
the feet of existing implementations.

Your attack should probably be documented as an example of how/why blind
retries can be dangerous though, in case some implementers would like to
go too far.

Regards,
Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Fri, 21 Jul 2017 10:31:46 -0500

> once, on the originally established connection).

(Some of this message seems addressed at Subodh and/or Martin, e.g.,
"your example".)
It may be helpful to distinguish between "replay" (attacker literally
resending the bits observed on the wire) and "retry" (attacker incudes
the client to send again the same plaintext but with different
crypto/bits on the wire).

So in some sense, I would characterize this as a "retry", as the client
is induced (by its timeout behavior) to resend the data.  Which set of
bits shows up first at the proxy/server does not seem as important to
me, for this terminology distinction.  The novel part of this attack is
that the proxy could be confused into indicating that the payload was
safe (er, into not indicating that it was unsafe, technically).  My
first instinct is that having the client always include the header is
the simplest remedy, and I'm not sure there's really a whole lot of harm
if the header still gets included on a retry that is no longer over 0-RTT.

> it's a well known problem for those of us having to implement persistent

I think that given the potential attacker abilities, we probably have to
assume that anything sent in 0-RTT is something that a clever attacker
could cause the client to retry.  That is, reiterating that only
safe/idempotent/side-effect-free messages should go in 0-RTT.

> retries can be dangerous though, in case some implementers would like to

I think it would be counterproductive to require that the client only
retry when given 4NN (too early), as that opens up a DoS vector that
would cause things to be much more sluggish than if the client had just
waited to send it as 1-RTT.

I agree that we could document this attack as an example of what could
happen and to reiterate that anything going in 0-RTT could is at risk of
both replay and retry.

-Ben

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 21 Jul 2017 15:53:01 +0000

I agree that we could document this attack as an example of what could happen and to reiterate that anything going in 0-RTT could is at risk of both replay and retry.

+1 for documenting the attack and dangers of sending things over 0-RTT, adding the header from the client, and moving forward with the draft.

Subodh

________________________________
From: Benjamin Kaduk <[email-masked]>
Sent: Friday, July 21, 2017 8:31:46 AM
To: Willy Tarreau
Cc: Martin Thomson; Subodh Iyengar; Mike Bishop; HTTP Working Group
Subject: Re: New Version Notification for draft-thomson-http-replay-00.txt

Hi Ben,

On 20 July 2017 at 06:25, Willy Tarreau <[email-masked]><mailto:[email-masked]> wrote:

I think that

(Some of this message seems addressed at Subodh and/or Martin, e.g., "your example".)
It may be helpful to distinguish between "replay" (attacker literally resending the bits observed on the wire) and "retry" (attacker incudes the client to send again the same plaintext but with different crypto/bits on the wire).

So in some sense, I would characterize this as a "retry", as the client is induced (by its timeout behavior) to resend the data.  Which set of bits shows up first at the proxy/server does not seem as important to me, for this terminology distinction.  The novel part of this attack is that the proxy could be confused into indicating that the payload was safe (er, into not indicating that it was unsafe, technically).  My first instinct is that having the client always include the header is the simplest remedy, and I'm not sure there's really a whole lot of harm if the header still gets included on a retry that is no longer over 0-RTT.

I think that given the potential attacker abilities, we probably have to assume that anything sent in 0-RTT is something that a clever attacker could cause the client to retry.  That is, reiterating that only safe/idempotent/side-effect-free messages should go in 0-RTT.

I think it would be counterproductive to require that the client only retry when given 4NN (too early), as that opens up a DoS vector that would cause things to be much more sluggish than if the client had just waited to send it as 1-RTT.

I agree that we could document this attack as an example of what could happen and to reiterate that anything going in 0-RTT could is at risk of both replay and retry.

-Ben

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 21 Jul 2017 18:23:53 +0200

Thank you for the explanation.

2017-07-19 16:49 GMT+02:00 Subodh Iyengar <[email-masked]>:
> A better way to think about it might be that mItm could always hold back the
> request even now, but he can't confuse the origin that the request came from
> 0-rtt or not 0-rtt because no such promise exists right now, however with
> this mechanism he can confuse the backend with a retry. So I think such an
> issue should be in scope for this discussion.

I do not think that the application will be confused.

For a backend, sending 4XX is a stateless thing. So it's just an issue
in which order the server sees the two requests confirmed with
ClientFinished (one that was sent by the client as a retry and the
other that was held back by an attacker). Since the two requests are
both ClientFinished-confirmed, to me it seems like that it is
identical to what could happen on TLS 1.2.

For example, to me it seems almost identical to an attack on TLS 1.2
that withholds the last packet that contains the HTTP request to
enforce the client to resend the request. The only difference would be
what timeout might kick in; in the TLS 1.2-case, a request timeout
would kick in if the attacker holds the last packet too long, whereas
in the 0-RTT case, it would be a TLS handshake timeout.

So while I think that it is an interesting attack (and that it is
worth mentioning in the document), I do not think that we need to move
the endpoint that sets the header to the client. The reason why I am
against such a move is that, moving it would make the worst case
performance of transactions using 0-RTT worse than when 1-RTT is used,
since the 4xx would cause a rentransmit in the entire chain.

> I don't think this mechanism is meant to stop replays, it's just meant to
> give the app the right information for it to make a decision of whether or
> not the client followed it's policy of sending data in 0-rtt. Replays are
> protected more by the transport rather than the application. For example if
> I had a transport without replay protection, this mechanism would not
> protect me from the classic replay safety attack of an attacker sending
> "safe" requests again and again and getting information from the length of
> the repsonse, if I was okay with the request being sent over 0-rtt.
>
> Subodh
>
>> HTTP
>>> hand-wavy.

-- 
Kazuho Oku

---

From: Ilari Liusvaara <[email-masked]>
Date: Fri, 21 Jul 2017 19:26:20 +0300

On Wed, Jul 19, 2017 at 02:49:32PM +0000, Subodh Iyengar wrote:
> A better way to think about it might be that mItm could always hold
> back the request even now, but he can't confuse the origin that the
> request came from 0-rtt or not 0-rtt because no such promise exists
> right now, however with this mechanism he can confuse the backend
> with a retry. So I think such an issue should be in scope for this
> discussion.

The attack you gave is not specific to 0-RTT or even to TLS 1.3.

To stop that sort of (retry) attack, the application must make POSTs
idempotent, even if HTTP says that POSTs are not idempotent. This
is even if 0-RTT is not implemented at all anywhere.

The 0-RTT data header, even in racy form that may misdetect 0-RTT
request as 1-RTT does stop actual replay attacks if server knows what
resources are sensitive. And thanks to difficulty of ideal anti-replay
at scale, the TLS 1.3 specification does admit replays.

-Ilari

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 21 Jul 2017 17:16:38 +0000

I agree, a generic retry attack is not specific to 0-RTT or even TLS 1.3.
However this draft adds a new header which has certain semantics.
The attack is just an example of how to use a retry to circumvent those semantics

and confuse the app server.

If the application does not use the header to decide anything and all it's

requests are idempotent, then does it really need the header at all? It could just accept

all requests. So clearly this header exists to make decisions based on it, could even be a

GET request it does not wish to serve over 0-RTT, or whatever.

that withholds the last packet that contains the HTTP request to
enforce the client to resend the request.

Ya its very similar, it's not even novel, I referred to it as a variant of dkg attack 😊.
The header adds new bit of information that the origin server is making a decision on. The thing that

this attack shows though is that that information can be faked in certain practical circumstances, which

makes it unsafe to rely on that information.

> make the worst case

Could you clarify this point Kazhuho?

Thanks,

Subodh

________________________________
From: [email-masked] <[email-masked]> on behalf of Ilari Liusvaara <[email-masked]>
Sent: Friday, July 21, 2017 9:26:20 AM
To: Subodh Iyengar
Cc: Kazuho Oku; Mike Bishop; Martin Thomson; HTTP Working Group
Subject: Re: New Version Notification for draft-thomson-http-replay-00.txt

-Ilari

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 21 Jul 2017 20:02:12 +0200

> Could you clarify this point Kazhuho?

Sure.

Http-replay is a specification that is required only when we have an
intermediary that terminates an HTTPS connection (a server application that
directly terminates TLS does not need a header, since it can look at the
state of the TLS stack).

So let's consider the case in which we are running an application server
behind a reverse proxy. The worst case would be when the two RTTs (one
between the client and reverse proxy and the other between the reverse
proxy and the server) are identical (the RTT is hereafter referred to as
RTTh; h standing for hop-by-hop).

If it was the case that the node that sets the early-data (which also is
the node that resends the request after 1-RTT confirmation when it sees a
4xx response) was the reverse proxy, then the use of 0-RTT and 4xx will not
increase any latency compared to the 1-RTT case. This is because the
reverse proxy will see 4xx at the same moment it sees ClientFinished. In
other words, the client will receive the response in 6 RTTh since it
started sending the TLS handshake.

OTOH, if we move the responsibility of setting the `early-data` header to
the client (and therefore require the client to do the resend), the 4xx
will need to go to the client, and then the request will travel though the
entire chain. This means that the client will need to wait for 8 RTTh since
it started sending the TLS handshake.

To summarize, by moving the responsibility of setting `early-data` header
to the client, we would see  33% additional latency (at worst) for using
0-RTT,  compared none in case of the original approach. And in addition to
that, we would see additional bandwidth consumed between the client and the
reverse proxy due to the retransmit of the request.

Thanks,
>
> Subodh
>
>
> ------------------------------
> *From:* [email-masked] <[email-masked]> on behalf of
> Ilari Liusvaara <[email-masked]>
> *Sent:* Friday, July 21, 2017 9:26:20 AM
> *To:* Subodh Iyengar
> *Cc:* Kazuho Oku; Mike Bishop; Martin Thomson; HTTP Working Group
> *Subject:* Re: New Version Notification for draft-thomson-http-replay-00.
> txt
>

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 21 Jul 2017 20:04:55 +0200

> reverse proxy due to the retransmit of the request.
Correction: the numbers for the original case is 3 RTTh, and the case for
the client setting the early-data header is 4 RTTh.

Sorry for the mistake.

>> *Subject:* Re: New Version Notification for

-- 
Kazuho Oku

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Jul 2017 14:47:59 +1000

> it started sending the TLS handshake.

That requirement was already present.  Though if your suggestion is to
allow an intermediary to retry a request, that's problematic in the sense
that it hides the fact from the user agent.  Sure, a user agent accepts the
risk of replay when it sends a request in early data, but that doesn't give
an intermediary a license to retry for it.

There's probably some way that we can describe rules for retrying at the
intermediary.  It does save a round trip between the user agent and
intermediary, and there is some potential value in having intermediaries
repair things.  But we lose some of the nicer end-to-end properties by
doing so.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 25 Jul 2017 07:22:12 +0200

> repair things.  But we lose some of the nicer end-to-end properties by

In fact I think there are some situations where it can safely be done,
specifically if the request was received as 1-RTT, or if the handshake
was completed in the mean time because it then becomes equivalent.

Probably that we should make a diagram to explain where and when the risk
of replay lies and make it more obvious what the intent of the retry is
(ie: prove that the request was not tampered with by using a safe
connection).

Willy

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Jul 2017 15:55:17 +1000

> was completed in the mean time because it then becomes equivalent.

Those are probably both cases where retries are inappropriate.

If the request arrives over 1-RTT, the intermediary probably shouldn't
be using 0-RTT.

If the connection was completed since the 0-RTT attempt was made, that
doesn't mean that it wasn't replayed already elsewhere.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 25 Jul 2017 08:26:29 +0200

> If the request arrives over 1-RTT, the intermediary probably shouldn't

I see your point. Some users may consider the intermediary as part of the
client, extending it to completely deal with retries, hiding them from the
client.

> doesn't mean that it wasn't replayed already elsewhere.

But that's the same if the client retries, which is why clients are encouraged
not to take too much risk by sending only apparently idempotent methods over
0-RTT.

Willy

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 25 Jul 2017 08:54:49 +0200

2017-07-25 13:47 GMT+09:00 Martin Thomson <[email-masked]>:
>> the client (and therefore require the client to do the resend), the 4xx will
>> need to go to the client, and then the request will travel though the entire
>> chain. This means that the client will need to wait for 8 RTTh since it
> That requirement was already present.

Oh. Thank you for pointing that out. I did not recognize that, but
rereading the draft, section 4.2 does seem to prohibit such behavior.

> Though if your suggestion is to allow
> an intermediary to retry a request, that's problematic in the sense that it
> hides the fact from the user agent.  Sure, a user agent accepts the risk of
> replay when it sends a request in early data, but that doesn't give an
> intermediary a license to retry for it.
>
> repair things.  But we lose some of the nicer end-to-end properties by doing
> so.

I think that the following model would be safe:

* consider 4NN as a signal that indicates the intermediary to wait for
1-RTT confirmation and retry
* 1-RTT confirmation can be observed as a logical and of
ClientFinished and non-existence of the early-data header

I do not see any reason to send 4NN to the initiator of the request
(i.e. browser), since the peer that the initiator talks to can always
use the ClientFinished as a confirmation. As I said, transmitting 4NN
to the initiator can impose additional latency, since contrary to
ClientFinished a client cannot (re)send a request until it sees the
response.

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 25 Jul 2017 09:29:47 +0200

> ClientFinished a client cannot (re)send a request until it sees the

To be precise, what I am suggesting here is that the server's action
for processing a request without replay-torelance can be defined as
follows:

* 0-RTT request with early-data header: send 4NN
* 0-RTT request with no early-data header: postpone request processing
until ClientFinished
* 1-RTT request with early-data header: send 4NN
* 1-RTT request with no early-data header: process the request

and that it can be deducted from the rule that when receiving a 0-RTT
request without an early-data header, an intermediary should: forward
the request to the origin by adding an early-data header, and if the
origin refuses to process the request by responding with 4NN, it
should wait (i.e. postpone the processing of the request) for
ClientFinished and resend the request to the origin server (this time
without the early-data header).

-- 
Kazuho Oku

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 25 Jul 2017 07:27:02 -0500

>> ClientFinished a client cannot (re)send a request until it sees the

That seems like it would violate the principle that the actual client
generating the request should have control over whether it is retried.

> for processing a request without replay-torelance can be defined as

I assume that by "without replay-torelance [sic]" you mean a request
that has potentially replay-unsafe consequences/side effects, so the
request must only be acted on over non-replayable transport.

> * 0-RTT request with early-data header: send 4NN

Yes.

> * 0-RTT request with no early-data header: postpone request processing

That seems permissible, but sending 4NN is also fine and requires less
state on the server.

> * 1-RTT request with early-data header: send 4NN

Yes.

> * 1-RTT request with no early-data header: process the request

Yes.

> without the early-data header).

This part I do not agree with, in particular the intermediary making the
decision to re-send the request without the early-data header.  I
believe that this decision must be left in the hands of the original
client, and do not think the latency concern justifies deviating from that.

-Ben

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 25 Jul 2017 14:36:17 +0200

> client, and do not think the latency concern justifies deviating from that.

In fact the client has no idea about the request's semantics nor safety,
only the application server does. The client may only approximate this
based on the method, the presence or not of a query string, etc... anything
that anyone else in the chain has access to and that is suboptimal. That's
why the 4NN generated by the server provides the best safety here : if there
is a risk that the server experiences a replay, it means it has accepted the
request carrying the Early-Data header, meaning it was replay-safe. Otherwise
the server would only process the only one without Early-Data.

Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 25 Jul 2017 08:41:21 -0500

> In fact the client has no idea about the request's semantics nor safety,

I think "no idea" is hyperbole, and maybe not helpful.  The application
server is the only one that has any hope of being authoritative, of
course (but could still be "confused about it" for certain types of side
channels).

> the server would only process the only one without Early-Data.

I'm not sure I'm understanding you properly, particularly with respect
to "experiences a replay".  Please also keep in mind the distinction of
terminology between replay and retry that I've been trying to enforce.

With respect to my original claim, I am thinking along the lines of
https://www.ietf.org/mail-archive/web/tls/current/msg23362.html where a
client might have external reasons to defer the retry behavior.  (This
is not necessarily the only use case, of course.)

-Ben

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 25 Jul 2017 17:30:27 +0200

Hi Benjamin,

Thank you for your response.

2017-07-25 21:27 GMT+09:00 Benjamin Kaduk <[email-masked]>:
> I assume that by "without replay-torelance [sic]" you mean a request that
> has potentially replay-unsafe consequences/side effects, so the request must
> only be acted on over non-replayable transport.
>
> That seems permissible, but sending 4NN is also fine and requires less state
> on the server.
>
> decision to re-send the request without the early-data header.  I believe
> that this decision must be left in the hands of the original client, and do
> not think the latency concern justifies deviating from that.

Could you please clarify the reason that an intermediary should not be
allowed to retry a request (that has once been rejected by 4NN) when
receiving an 1-RTT confirmation from client, without setting the
early-data header?

To me, the behavior is identical to a server that postpones a request
(that was received as 0-RTT data) until it sees a ClientFinished.
Which is a behavior you state as permissible.

-- 
Kazuho Oku

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 25 Jul 2017 11:06:51 -0500

> Which is a behavior you state as permissible.

In the general case, the request can be different when generated for the
retry versus the initial request.  This is quite obvious for the token
binding draft proposal in
https://tools.ietf.org/html/draft-ietf-tokbind-tls13-0rtt-02 but I doubt
that's the only possible case.  Granted, the token binding case is not
especially applicable to the proxy case being discussed here since the
proxy would need to be involved in handling token binding for that to
work.  But I hope it helps to illustrate that the client should retain
control over how the request is retried and that the proxy may not have
all the information necessary to automatically retry.

-Ben

---

From: Martin Thomson <[email-masked]>
Date: Wed, 26 Jul 2017 10:26:56 +1000

> That seems permissible, but sending 4NN is also fine and requires less state

This is our "legacy" case.  That is, clients that do 0-RTT today
without this new mechanism.  The cost to sending 4NN is that it causes
the client to use generic 400 processing, which probably doesn't
include a retry.  That's not the end of the world, though it probably
means that the response is a terminal failure as opposed to a
recoverable one.

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 26 Jul 2017 10:30:23 +0900

Hi Benjamin,

2017-07-26 1:06 GMT+09:00 Benjamin Kaduk <[email-masked]>:
> especially applicable to the proxy case being discussed here since the proxy
> would need to be involved in handling token binding for that to work.  But I
> hope it helps to illustrate that the client should retain control over how
> the request is retried and that the proxy may not have all the information
> necessary to automatically retry.

Thank you for the example.

I can understand that the request transmitted in 0-RTT and 1-RTT can
be different. However I do not understand why the fact leads to a
conclusion that the client should be the one to resend the request.

My understanding is that in Token Binding either a secret derived from
early_exporter_master_secret or exporter_master_secret is used. It is
true that there is no replay resistance when the server receives 0-RTT
data with Token Binding. However, replay resistance is gained at the
moment the server receives (and successfully validates) a
ClientFinished. By using ClientFinished as a confirmation, there is no
need to request the client to resend the request, even if Token
Binding was used.

I believe that all data that are protected using early secrets gain
replay resistance at the moment the TLS handshake completes. Hence no
need to require the client to resend data.

-- 
Kazuho Oku

---

From: Martin Thomson <[email-masked]>
Date: Wed, 26 Jul 2017 11:46:13 +1000

I'm inclined to side with Ben on this one.  The principles I've been
using to assess this design is mutual consent.  If the intermediary
retries a request without the knowledge of the client, that violates
that principle.  A retry will look like a completely new request when
it hits the origin and it therefore won't trigger any special defense
mechanisms.  The main reason to let the client do the retry is to
allow it to reassess the risks before retrying.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 26 Jul 2017 05:44:18 +0200

> allow it to reassess the risks before retrying.

In fact, while I personally have no interest in having the intermediary
perform a retry, I understand Kazuho's point of view here. It's a matter
of the intermediary's role. If we cut this into "systems", we can see
this :

       +---------+    0-RTT      +--------+
       |  client +---~~~~~~~~~---+ server |
       +---------+               +--------+

Here a server considering the request unsafe for 0-RTT will wait for
ClientFinished. It's not particularly defined how the server is
architected here. It can be a TLS termination followed by a clear
server, it can be a multi-layer application server with the first
layer asking a database whether or not such a request is considered
safe to pass to the next stage or if it must wait for ClientFinished.

But it could also be a front gateway serving as the only entry point,
keeping the connection with the client and politely asking the server
"do you think this request is safe enough to process or do you prefer
me to wait for ClientFinished ?". We can view our proposal as such a
mechanism in the end :

                   +-------------------------+
                   | +---------+  +--------+ |
                   | |  edge   |  | origin | |
             ------+~| gateway |~~| server | |
                   | +---------+  +--------+ |
                   +-------------------------+
                      "server"

  Edge Gateway (part of the "server" system above) :
     "Dear server, I've received this request involving Early Data,
      do you want to process it now or do you prefer me to wait for
      ClientFinished first ?"
     (this is notified via "Early-Data: 1")

  Origin Server :
     "Oh don't bother, it was safe enough and I processed it, here's
      the response"
     (this is notified via 2xx/3xx status code)
    or :
     "Well, I can't process it like this, I need the ClientFinished
      first to ensure it's not subject to a replay attack"
     (this is notified via the 4NN too early status code)

In the latter case, it totally makes sense to imagine that the intermediary
waits for the ClientFinished and passes the request to the server because
it got a guarantee by the server that this request was not processed and
that any possible rogue copy of that request hasn't been processed either
and will not be.

So while we designed the 4NN principle as an end-to-end invitation to
retry, offering the guarantee that the request was not processed and
that it will not be replayed, this same signaling is perfectly usable
between components. The fact that the client isn't notified of the
intermediary retrying it is not a problem because if the client has
sent its request in 0-RTT and has sent its ClientFinished, it's because
it expected it to be completed. By the way that was the recommended way
to implement 0-RTT before we discussed this at the workshop ;-)

However, I'm not willing to open the draft to let intermediaries retry
in the general case, at least for the reasons Ben mentionned and because
for me there is a difference between the specific model above where we
consider the intermediary as part of the server system and many other
models. I'd rather not see blackboxes decide to do whatever on the
client's behalf as a way to "accelerate" networks...

But I think that by using a careful wording we can ensure that implementers
who consider themselves as part of the server system see what to do in their
situation. We can possibly put a few MUST NOT rules for specific cases which
risk to become border-line.

Regards,
Willy

---

From: Martin Thomson <[email-masked]>
Date: Wed, 26 Jul 2017 14:19:29 +1000

I find this pretty convincing. But...

Now we get to the technical hurdles (and I keep forgetting this one,
so I apologize).

If the intermediary is the first, then this design is fine.  However,
if there is more than one intermediary, unless the intermediary in
question is immediately adjacent to the client, it can't be sure that
the Early-Data header field wasn't present because an earlier hop
added it.   Early-Data is permission to forward a request in 0-RTT
also, so you could wait on this hop, but the handshake on the previous
hop might not have completed.

I don't know how to solve for that case without a whole lot more
machinery.  I don't know if it's a problem.  Do people nest CDNs?  The
CDNI working group would seem to suggest that some do, or at least
plan to.

If Early-Data was omitted by the client, that would make it easier in
a sense.  Then an intermediary could tell if it was the first.  But
then we're back to the confusion problem Subodh described.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 27 Jul 2017 09:08:08 +0200

Hi Martin,

On Wed, Jul 26, 2017 at 02:19:29PM +1000, Martin Thomson wrote:
> a sense.  Then an intermediary could tell if it was the first.

If you remember that's exactly the conclusion that draw us to get rid
of this header on the client during our first meeting. We noticed that
it was wrong to have the client provide it because it would confuse
chained intermediaries and in the end the only thing that matters is
not how the request was *sent* but how it was *received*.

Willy

---

From: Martin Thomson <[email-masked]>
Date: Thu, 27 Jul 2017 17:15:20 +1000

Hi Willy,

> not how the request was *sent* but how it was *received*.

Yes, this was a good reason until Subodh convinced me that the race
was serious and that how the packet is received can be controlled by
an attacker to the extent necessary to confuse a server.

It's true that receipt is what matters ultimately.

I've updated the PR
(https://github.com/martinthomson/http-replay/pull/25) to capture this
nuance.  Hopefully it isn't awful.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 27 Jul 2017 10:31:41 +0200

Hi Martin,

> an attacker to the extent necessary to confuse a server.

I continue to disagree on this specific case because it's exactly the
same as what can be done without 0-RTT, and 0-RTT doesn't make it any
weaker.

We're "just" talking about a generic way to abuse a supposed ability
for a client to retry upon network timeouts. This exists in clear, in
TLS with and without 0-RTT. The attacker can even just block the
response from being delivered to the client, to ensure it's really
handled. All that's done here is to provoke a timeout and watch what
happens.

The draft never suggests at all that a client should retry on timeouts
met with 0-RTT, it suggests the client has to retry when getting the
4NN response, promising the request was not processed.

> nuance.  Hopefully it isn't awful.

I really disagree with one this because it removes the ability to wait
for the ClientFinished signal for all communications, even with client
talking directly to the server. It significantly affects the usage scope
of 0-RTT in my opinion, to try to cover for a case that is already not
covered using 1-RTT.

I'd rather remind of the risks of retrying a failed request upon timeout
instead of waiting for the proof of non-execution that 4NN consists in, but
that covers a broader scope, more in line with Mark's initial enumeration
of all retry cases.

Best regards,
Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 27 Jul 2017 13:10:45 -0500

> same as what can be done without 0-RTT, and 0-RTT doesn't make it any

I'm not sure that it's "exactly" the same, though perhaps the degree of
difference is only qualitative.  With a well-behaving client and server,
and an attacker in the network, my qualitative expectation is that the
attacker can cause a given request to be processed once as 0-RTT
(actually more in many setups) and once as 1-RTT.  What's different in
Subodh's attack is that the attacker can cause the request to be
processed twice as 1-RTT (and maybe more as 0-RTT), so the number of
1-RTT processings has gone up.

Of course, an attacker doing the analogous thing when TLS 1.2 is in use
would also cause the request in question to be processed twice as 1-RTT,
since there is no 0-RTT available!  So, the main qualitative difference
that seems new here is the server being convinced it did something
"safer" than it actually did.  I've not fully thought out the
implications of this, whether there are plausible scenarios that could
lead to (e.g.) money being spent twice, etc.

> of 0-RTT in my opinion, to try to cover for a case that is already not

I think that one of the key interactions here is that TLS itself is a
two-party protocol, but for this document we are forced to consider
multi-party interactions (chains of TLS connections).  So for any given
TLS connection, waiting for ClientFinished is safe, but the composition
is not, which is where we can get into trouble.

Just tossing a random idea out there -- how crazy would it be to have
the client send Early-Data: 1, but each successive hop incrementing the
value, so we get Early-Data: 2, Early-Data: 3, etc. indicating the
"nesting level"?  That would perhaps permit some of the optimizations
we've talked about, like waiting for ClientFinished when talking
directly to the client.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 27 Jul 2017 13:18:50 -0500

> to implement 0-RTT before we discussed this at the workshop ;-)

I just want to point out that 4NN was not a *guarantee* that the request
will not be replayed until
https://github.com/martinthomson/http-replay/pull/25 added text that the
server MUST consistently use the same approach, if I understand things
correctly.  That is, draft-thomson-http-replay-00 leaves open the
possibility for one server in a cluster to return 4NN and another server
in that cluster to have a different level of load (or whatever) and
successfully process the request.

> risk to become border-line.

I agree with Martin that this is pretty convincing, and hope we can come
up with appropriately flexible language that maintains the other
properties we're looking for.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 27 Jul 2017 13:32:32 -0500

> conclusion that the client should be the one to resend the request.

The client may be the only one that has all the information needed to
construct a proper 1-RTT request.

> need to require the client to resend data.

I think your use of the phrase "gain replay resistance" is confusing and
perhaps misleading.  Anything sent via 0-RTT is potentially subject to
replay, and nothing can change that.  What happens at receipt of
ClientFinished is an indication that *this particular instance* being
received by the server is not a replay.  But the same data could be
replayed to, and potentially accepted at, other servers as well.

-Ben

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 27 Jul 2017 20:34:37 +0200

On Thu, Jul 27, 2017 at 01:18:50PM -0500, Benjamin Kaduk wrote:
> will not be replayed

Yes it is. The only way to get this 4NN is for the connection to have
been maintained between the client and the server for the whole duration.
An attacker duplicating the traffic cannot use it against a new connection
because the ClientFinished depends on the handshake.

That's why I'm insisting that the 4NN *is* a guarantee, that the timeout
isn't. Subodh's attack relies on letting the client connection time out
before receiving anything in hope that the client would retry it, leaving
a working connection to the attacker.

Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 27 Jul 2017 13:45:34 -0500

> because the ClientFinished depends on the handshake.

I think we may be using different vocabulary/definitions.

Alice makes a TCP connection to Bob, starts up a TLS 1.3 handshake and
sends some early data with it containing an HTTP request.  Eve has a tap
in the network and records the bits going Alice-->Bob.  Bob sends a TLS
1.3 ServerHello/EncryptedExtensions/Finished and a 4NN in its 0.5-RTT
data.  Nothing stops Eve from taking a copy of those recorded bits and
sending them towards Bob's clone Robert that shares a cert+key.  Are
there requirements on Robert's behavior when receiving those bits?  I
claim that with the -00, Robert is permitted to give 200 and an actual
response, but with pull/25, Robert is obliged to also return a 4NN.

Again, by "replay" I mean literally replaying the bits on the wire, not
a retry where the client generates a new request.

> a working connection to the attacker.

A guarantee of what, exactly?  (I assume only upon receipt, since the
attacker can withhold the response from the client and force timeout
behavior.)

-Ben

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 27 Jul 2017 21:50:14 +0200

> response, but with pull/25, Robert is obliged to also return a 4NN.

I don't see how this would make any difference. In before pull/25, both
Alice and Eve send the request as early data without ClientFinished. This
*without ClientFinished* is the signal that the request is at risk. After
pull/25, both Alice and Eve send the request as early data without
ClientFinished and both with the EarlyData header field. This one becomes
the signal. In both cases, it requires both Bob and Robert to have the
same configuration, so nothing was saved by adding EarlyData.

The point that Subodh made was that Bob would hijack Alice->Bob's connection
and would delay the ClientFinished message so that the finalized request
doesn't reach Bob. Alice, being fed up with waiting would *spontaneously*
try again and succeed. Then Eve would unblock the packet and have Bob play
the initial request again. The fault here is not a lack of identification
that the first request was transported over early data, it's a fault from
Alice who retried without being invited to do so nor knowing the consequences
of doing so. And if this attack would work for a selection of Alices, then it
would also work over TLS 1.2 as well because no transport information was used
at all.

> a retry where the client generates a new request.

We do uses the same terminology, but you and Subodh are talking about
different cases.

> attacker can withhold the response from the client and force timeout

A guarantee that the request that led to this response was not processed.
The timeout behaviour is made up and not part of the draft at all, and is
irrelevant to 0-RTT.

Hoping this helps,
Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 27 Jul 2017 15:16:28 -0500

> same configuration, so nothing was saved by adding EarlyData.

My claim is with regarding Bob/Robert's decision about whether a given
(early) request can safely be processed as early data (vs. needing to be
verified by waiting for ClientFinished or sending 4NN for 1-RTT retry)
-- that is, whether they must necessarily make the same decision for a
given request.  You mention that they must have the same configuration,
but but what if the current system load also is used in making the
decision?  My reading of pull/24 is that "MUST consistently use the same
approach" also implies must make the same decision.  If you think that
the -00 draft as-is implies that Robert and Bob must make the same
decision, I'd be happy to have that be the case.

> would also work over TLS 1.2 as well because no transport information was used

My understanding was that Subodh's point involve delaying not just the
ClientFinished but also the early data preceding it, so that Bob did not
even know there was a request at all, let alone one that is not
finalized.  When the early data + ClientFinished were allowed through
together, Bob would see the request, check that the handshake is
finished, and assume it was a 1-RTT request, even though Alice had
already timed out and retried.

I do not dispute that Alice could prevent the behavior by waiting to
receive a 4NN before retrying, though I also do not expect (e.g.)
browsers to universally adopt such behavior.

> The timeout behaviour is made up and not part of the draft at all, and is

I agree that a client receiving 4NN has a guarantee that its request was
not processed on this connection.  Does it also have a guarantee that
(replays of) its request were not processed on any other connection
[even though those other connections would subsequently have the TLS
handshake fail]?

-Ben

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 27 Jul 2017 23:01:20 +0200

On Thu, Jul 27, 2017 at 03:16:28PM -0500, Benjamin Kaduk wrote:
> decision?

It doesn't change the fact that the additional presence of the EarlyData
header field added by the client makes no distinction here, it's just
redundant with the connection using 0-RTT.

> My reading of pull/24 is that "MUST consistently use the same

Having both of them rely on EarlyData demands that same for of consistency
unfortunately so it doesn't add anything against this.

I mean, don't get me wrong, I'm not saying at all that there are not corner
cases that we have to try to improve, what I'm saying is that the proposal
to have the client add Early-Data was proposed solely for the timeout case
and provides zero benefit for the case where the early data are replayed
without ClientFinished (thus without the client being aware), and that's
precisely the limit of using 0-RTT, so if the whether *one* of the servers
considers the request safe or not (configuration issue or anything) is out
of our control. The principle is to let the server decide what action to
take with requests received as Early Data.

> finalized.

Not exactly since the Early Data has to be sent before the ClientFinished,
the point was to be able to present Bob with a request carrying a validated
ClientFinished (ie: Bob was supposed to trust it).

> When the early data + ClientFinished were allowed through

Yep.

> browsers to universally adopt such behavior.

But the point of the draft precisely is to explain how to safely map
HTTP on top of early data and to explain the risks :-) If we start to
say "let's not do it safely because some will surely bypass the rule
anyway", it becomes a bit difficult to provide something really useful :-)

That's why I'm really all for documenting the corner cases so that
implementers know what *not* to do (ie: even if a browser retries on
timeout, it must absolutely not do it with early data). And anyway as
Subodh showed, a browser retrying on timeout would be at risk whatever
the transport level in use.

> [even though those other connections would subsequently have the TLS

Absolutely not, but it does have a guarantee that this request was not
presented with a ClientFinished. So by virtue of the server having the
final choice whether a request is safe to process as early data or not,
the second will fail. The remaining cases where such a request is
nevertheless processed are caused by different configurations, but adding
headers on the client will not address misconfigured servers anyway (and
would in fact make things worse by increasing the complexity, resulting
in additional wrong combinations statically speaking).

Willy

---

From: Martin Thomson <[email-masked]>
Date: Fri, 28 Jul 2017 09:37:37 +1000

> redundant with the connection using 0-RTT.

This is the core of the issue.  And this statement is true.  If the
data arrives in early data - and that can be detected reliably by a
recipient - then the header field doesn't add any value.

The point on which we seem to be disagreeing is whether it is better
to increase the reliability of the signal (by mandating use of
Early-Data by clients), or to insist that server (or serving
intermediary) implementations are careful in determining whether a
request arrived in (or partially in) early data.

The key advantage of NOT mandating Early-Data is that it enables a
strategy of waiting.  An absent header field means that this is the
first hop to use early data.  Mandating use of the header field means
that a server cannot know whether there was a previous hop that used
early data and so it cannot wait for the client Finished (that
previous hop might not have completed correctly).

The key advantage of mandating Early-Data is that it improves the
signal.  But it removes the waiting option from consideration (that's
a bug in my PR).

After some consideration, I'm somewhat more inclined to side with
Willy on this particular issue.  Mainly for the performance advantage,
but also because it makes the default implementation easier (no
modification of header fields on retry).  I will acknowledge that it
makes server implementations more difficult; a lot more hinges on
whether the server is able to reliably detect that a request was
received in early data.

The requirement to treat requests consistently is something we should
retain no matter what the outcome of this discussion is.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 28 Jul 2017 06:32:57 +0200

> retain no matter what the outcome of this discussion is.

This can be one of the strong prerequisites in the spec. After all the
problem is generalized. Early data come with high risks of replay and
every user need to be careful when using them, both clients which should
only use it for safe requests, and servers which need to be prepared to
handle it safely. The reason I refused to implement support for this in
haproxy precisely is because of the lack of a strict policy around it
making it dangerous. If the text is strong enough and the risks are
clear, it can really be a great feature.

Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Fri, 28 Jul 2017 07:58:33 -0500

> recipient - then the header field doesn't add any value.

Yes, that statement is true.

> request arrived in (or partially in) early data.

Sure.  And I think some of the viewpoints have been driven by existing
implementations that only offer a "is the handshake finished?" API where
it's harder to tell "did any part of this request come in over early data?".

> previous hop might not have completed correctly).

(No comments on the "increment the number in the Early-Data header on
each hop" strawman yet?)

> modification of header fields on retry).  I will acknowledge that it

I'm certainly willing to be part of a consensus that allows the waiting
option...

> whether the server is able to reliably detect that a request was

... but we'll need to make sure that TLS stacks provide useful APIs for
servers to be able to do this.

Just to be clear, we're ending up in a space that allows waiting for the
first hop, but requires subsequent hops to send the 4NN if any part of
the request came in over early data, is that correct?  (That is,
later-than-the-first hops do not have the option of waiting for
ClientFinished.)

> retain no matter what the outcome of this discussion is.

Yay!

More replies to Willy's previous mail separately, though I don't think
they affect the question at hand.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Fri, 28 Jul 2017 08:27:40 -0500

I think we've been at least partially talking past each other, and I can
take the blame for some of that -- you seem to have been trying to keep
a narrow focus on the question of whether the Early-Data header is sent
on the first hop, and I've been expounding on various other potential
issues in the 0-RTT ecosystem that are not really relevant to that more
narrow question.

> redundant with the connection using 0-RTT.

Yes (as covered in the other subthread).

> unfortunately so it doesn't add anything against this.

Both would have to know the request came in early data, whether via the
header or a direct query to the TLS stack, yes.  But that was something
of a fundamental prior assumption and not the "interesting" part of my
hypothetical, which was intended to be covering the decisions Robert/Bob
made when inspecting the body of the request (in the context of knowing
it was in early data).  But it sounds like we're happy to require
consistency from multi-server pools in this regard so I don't need to
continue trying/failing to expound about it.

> without ClientFinished (thus without the client being aware), and that's

Agreed

> take with requests received as Early Data.

I'm not sure I understand what you mean by "precisely the limit of using
0-RTT".  If I can attempt to rephrase what you're saying here it would
be that "we need to make sure that the server knows a given request was
(partially) received as early data (at some point in the chain), and the
server gets to decide whether it's safe enough to reply to right away. 
We can cover the consistency of behavior across servers as well, but
that is a separate issue."

> ClientFinished (ie: Bob was supposed to trust it).

The point was to be able to present Bob with such a request, yes.  But
we also require that Bob does not just go ahead and process the request
without the ClientFinished, since then he would not be treating it as
validated early data.
Thinking about it more (and without presuming a specific set of
requirements of the many we are considering), the necessary situation
could be obtained in at least two ways: (1) blocking both the early data
and the ClientFinished, or (2) only blocking the ClientFinished and
relying on Bob using the strategy of "wait for ClientFinished before
processing the request".  In my previous mail I was only considering (1)
to the exclusion of (2); sorry about that.

> anyway", it becomes a bit difficult to provide something really useful :-)

Yes, let's avoid "let's not do it safely because some will surely bypass
the rule
anyway" :)

> the second will fail. The remaining cases where such a request is

Saying "the second will fail" without qualifications makes me a little
nervous, as it depends on the server's behavior/the consistency across
servers.  But I think I see what you are getting at, and agree.

> in additional wrong combinations statically speaking).

I agree that adding more headers on the client won't help with
misconfigured servers.

-Ben

---

From: Martin Thomson <[email-masked]>
Date: Mon, 31 Jul 2017 10:07:45 +1000

> it's harder to tell "did any part of this request come in over early data?".

That might be the case.  The perception that this API thing is some
sort of fundamental disagreement is odd.  You might yet convince me
that we need something, at least on the server side.

I have been convinced that a server application needs greater
certainty than the simple "handshake is finished" signal.  We might
actually have that certainty, but I want to double-check.  I need to
do some investigation on our implementation before I comment more, but
I'll get back to you on this.

> (No comments on the "increment the number in the Early-Data header on each
> hop" strawman yet?)

That might work.  If our model includes participants in hops that
understand early data that always increment the value (both on the
send and receive side).  The only place where it won't increment is if
an intermediary doesn't understand Early-Data, but that's OK because
they won't be using it.

> first hop, but requires subsequent hops to send the 4NN if any part of the
> request came in over early data, is that correct?  (That is,

That would be the plan.  The only gotcha being whether we can have a
reliable signal to the server.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 31 Jul 2017 05:43:24 +0200

Hi!

> I'll get back to you on this.

In fact from the various discussions we've had around early data, it
seems mandatory that an API exposes one way or another the fact that
*some* such data were replay-unsafe if 0-RTT has to be implemented.

> an intermediary doesn't understand Early-Data, but that's OK because

That's very close to our initial model consisting in having 0, 1 and 2
typically for "none, sent, received", that we found problematic when
chaining multiple intermediaries because in case the 0-RTT was used
again further down the path it was a bit confusing. My point was that
it was important to distinguish between the fact that a client attemped
to use early-data and the fact that a server received it this way.

I think that during this brainstorming we shoud avoid to think about
"cheap" models that often break against one example. If instead we
temporarily consider the Early-Data header field as a list of states
having each 2 values ("sent", "received"), and that each sender has
to append "sent" to the list and each receiver has to append "received",
we can end up with something like this on the server side in Kazuho's
case where an intermediary uses early-data to the server after using
it with the client for example:

   Early-Data: sent, received, sent, received

At least the server gets all the information about what happend on
the path. Now the question is : what should the server do with such
information, what is useful there and how could it be simplified.

If we have the confirmation that TLS APIs provide the information that
at least some data were received as early data before ClientFinished,
then we can remove the "sent" list (that said, APIs evolve and if we
make it mandatory to have that signal in order to map HTTP on top of
TLS 1.3 0-RTT, it will exist).

Early-Data then becomes a list of "received" fields and we'll note
that a server is interested in seeing at least one entry, which comes
back to the current proposal of "were some early-data received anywhere
along the path before a ClientFinished, tagging the request as
potentially unsafe ?", leaving the responsibility to the first such
receiver on the path to tag the request before passing it.

By the way there's something I actually like with the current proposed
method of "tagging" a request as unsafe where it was first noticed : it
doesn't require the ability anywhere to modify an existing header's
contents at all, reducing the temptation for implementers to "untag"
a request that was tagged by a previous intermediary. Using a counter
requires such a change and we could expect some bugs where an intermediary
retrying a connection using 1-RTT after having tried 0-RTT would decrement
the header's content to indicate that finally it's not sending this way.
And if it retries multiple times and is poorly implemented it could even
remove some of the previous counts, which I don't really like :-/

> reliable signal to the server.

Yep, which shows the importance of the first hop making use of some
Early-Data here in this chain. If we called this header field "untrusted"
or "unsafe" it would be obvious that it's the first receiver's responsibility
to put such a stamp on such a request.

Cheers,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 31 Jul 2017 06:02:07 +0200

Hi Ben,

> issues in the 0-RTT ecosystem that are not really relevant to that more

No problem and no blame to get here. The subject is complex and it's
important to ensure that the model is reliable and provides some value.

Yep.

> that is a separate issue."

No we can't even cover it, it has to be a prerequisite for implementation.
In fact we're documenting how to use a risky but very appealing feature
safely with HTTP and posing some elements so that it doesn't represent a
risk anymore. When we write "the server must do this", if it's a server
farm and not a single node, they all have to act consistently. If there
is a risk of inconsistency past a certain point, it becomes the front
intermediary's role to either respond with 4NN or wait for ClientFinished
to ensure reliability before the uncertain path.

I think it's very important to document a number of risks to be clear
about whether or not 0-RTT can be safe to use in certain deployments.
Many users have no idea about these risks and are willing to put it
anywhere without any check. I've been asked repeatedly "when will haproxy
implement 0-RTT", and I always replied "never until it's safe to use
with HTTP".  Then "but others are already using it so surely it's safe"...
Here we know it's not always the case and that it must be clear when it
is and when it is not. With appropriate documentation, implementers will
know that they'll do 0-RTT till the gateway, wait for ClientFinished here
and always protect unsafe servers. Others will do this only for application
servers and will let requests for static contents to go to the static farm
with only the early-data header field set because the risks will be
understood.

> to the exclusion of (2); sorry about that.

No pb. But (1) doesn't work. You can't block early data *and* ClientFinished
because ClientFinished comes after the RTT as a response to the server
handshake, which will not come before the early data are sent to the server :-)

> servers.  But I think I see what you are getting at, and agree.

Great ;-) Keep in mind that our responsibility is to write "if you can't
ensure consistency across your farm, the intermediary fronting this farm
must return 4NN".

Cheers,
Willy

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 31 Jul 2017 10:36:02 -0500

Hi Willy,

> No we can't even cover it, it has to be a prerequisite for implementation.

Okay, I'm happy to support that :)

> to ensure reliability before the uncertain path.

Just to double-check, we will still have text that says the server farm
must act consistently (as opposed to just saying "the server must do
this" and making it implicit that a server-cum-server-farm is bound to
behave as if a single server)?

> implement 0-RTT", and I always replied "never until it's safe to use

This is very reassuring for me to hear, thank you.

> handshake, which will not come before the early data are sent to the server :-)

The server is supposed to send its
ServerHello/EncryptedExtensions/Finished immediately after it sees the
ClientHello, and the client can keep streaming early data while that's
happening.  The client should send EndOfEarlyData/Finished once it
receives the server's Finished, and move the data stream to 1-RTT data. 
So I think that the attacker can let through the ClientHello but hold on
to the application data until the server's first flight arrives, then
let through the stored records along with the client's response, all in
one go (possibly with additional delaying for the whole assembly).

> ensure consistency across your farm, the intermediary fronting this farm

Sounds good.

Thanks,

Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 31 Jul 2017 10:59:35 -0500

Greetings!

> *some* such data were replay-unsafe if 0-RTT has to be implemented.

I think so, yes.  Expounding on my previous comment about two-party vs.
multi-party interactions, it seems that the HTTP case is qualitatively
different than what we had previously been thinking about in the TLS WG
for an abstract use case.  It seems that we are not just interested in
whether a given TLS message is replayable or is a replay, we rather want
to know whether the contained HTTP message could be replayable in a
global system.  Even if the current TLS message is definitively not
itself a replay (by virtue of ClientFinished validation), that TLS
message might be replayable at a different part of the server farm
(preventable by server implementation), or the TLS message from a
previous hop (containing the same HTTP message) could be replayable, and
that's the part that we don't locally have control over.  So when the
intermediary goes forward, if the message it received was replayable,
then the intermediary must include an indication of that fact, so that
the origin server can send 4NN if the replayability (which in this case
is not controllable locally by the origin server) is unacceptable.

There seem to be two follow-ups from this: (1) if the intermediary knows
that it is part of a system that uses a globally strong replay cache, it
seems like it might still be safe for the intermediary to use 0-RTT
and/or skip the Early-Data header when relaying the request to the
origin.  (But I still want to think about that more.)  (2) It seems that
we may only need this behavior if the *entire* HTTP request came in over
early data, and that requiring this behavior if *any* part of it came in
via early data may be overkill.  That said, I'm not convinced that it's
easy to robustly implement an API for that in all TLS stacks, so going
with the stronger requirement may be safest.

> remove some of the previous counts, which I don't really like :-/

Thanks for cluing me in on the history.  Your reasoning seems sound, and
I no longer think that the incrementing counter could provide real value
over the current proposal.

-Ben