Subject: Consensus Calls for Transport/TLS issues, post-Cupertino

From: Mark Nottingham <[email-masked]>
Date: Tue, 22 Oct 2019 11:42:13 +1100

The following issues have proposals for resolution, and discussion so far seems to support consensus to accept them. If you object, please do so on the issue or in response to this message (changing the Subject appropriately!). Absent any pushback, we'll direct the editors to incorporate them next week.

See <https://github.com/quicwg/base-drafts/projects/5> for the current state of issues in the Late Stage process, itself defined at <https://github.com/quicwg/base-drafts/blob/master/CONTRIBUTING.md>.

* #3097: Is CONNECTION_CLOSE ACK-eliciting?
   The proposal is <https://github.com/quicwg/base-drafts/issues/3097>

* #3085: Stateless reset detection should be datagram-based
   The proposal is <https://github.com/quicwg/base-drafts/pull/2993>

* #3054: Label for key updates
   The proposal is <https://github.com/quicwg/base-drafts/pull/3050>

* #3046: Handling of Retire Prior To field
   The proposal is <https://github.com/quicwg/base-drafts/pull/3096>

* #3037: Require peers to check if RETIRE_CONNECTION_ID sequence number is valid
   The proposal is <https://github.com/quicwg/base-drafts/pull/3036>

* #3027: Codes for frame encoding errors
   The proposal is <https://github.com/quicwg/base-drafts/pull/3042>

* #2944: Layout of PreferredAddress
   The proposal is to close with no action.

* #2928: Lift single-packet ClientHello requirement?
   The proposal is <https://github.com/quicwg/base-drafts/pull/3045>

* #2863: unrecoverable loss pattern leads to deadlock
   The proposal is <https://github.com/quicwg/base-drafts/pull/3121>
   
* #2823: Do Initial secrets change after Retry packet?
   The proposal is <https://github.com/quicwg/base-drafts/pull/2870>

* #2741: Re-visit initial keys discard
   The proposal is to close with no action.

* #2152: Why does stateless reset have to be checked after MAC failure
   The proposal is <https://github.com/quicwg/base-drafts/pull/2993>

--
Mark Nottingham   https://www.mnot.net/

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 22 Oct 2019 10:31:11 +0900

2019年10月22日(火) 9:42 Mark Nottingham <[email-masked]>:

> The following issues have proposals for resolution, and discussion so far
> seems to support consensus to accept them. If you object, please do so on
> the issue or in response to this message (changing the Subject
> appropriately!). Absent any pushback, we'll direct the editors to
> incorporate them next week.
>
> See <https://github.com/quicwg/base-drafts/projects/5> for the current
> state of issues in the Late Stage process, itself defined at <
> https://github.com/quicwg/base-drafts/blob/master/CONTRIBUTING.md>.
>

I feel sorry but I object to the proposed approach. As I have pointed out
in my comments in #2863, not discarding the Handshake key forever breaks
our assumption that no Handshake packets will be sent or received once the
handshake is confirmed. We missed that there are these side effects when we
rush to the conclusion at the end of the interim.

Specifically, we'd be required to define how Handshake packets deal with
migration (does the path for Handshake packets change? If it does, how do
we fill in the SCID field of the Handshake packet? If it does not, until
when do the endpoints need to retain the original path?).

I think we should fix the deadlock, rather than changing other parts of the
protocol that currently depend on the Handshake key being discarded at some
point.

-- 
Kazuho Oku

---

From: Nick Banks <[email-masked]>
Date: Tue, 22 Oct 2019 01:39:22 +0000

I'd also prefer to fix the problem, even if it means bringing back something like RETIRE_KEY.

- Nick

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: QUIC <[email-masked]> on behalf of Kazuho Oku <[email-masked]>
Sent: Monday, October 21, 2019 6:31:11 PM
To: Mark Nottingham <[email-masked]>
Cc: Lars Eggert <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

2019年10月22日(火) 9:42 Mark Nottingham <[email-masked]<mailto:[email-masked]>>:
The following issues have proposals for resolution, and discussion so far seems to support consensus to accept them. If you object, please do so on the issue or in response to this message (changing the Subject appropriately!). Absent any pushback, we'll direct the editors to incorporate them next week.

See <https://github.com/quicwg/base-drafts/projects/5<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fprojects%2F5&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982835221&sdata=jB%2B7jce%2F4Qj7CRUvUK7OQBLcZwUiQ2ugPJNA2SnDlZ0%3D&reserved=0>> for the current state of issues in the Late Stage process, itself defined at <https://github.com/quicwg/base-drafts/blob/master/CONTRIBUTING.md<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fblob%2Fmaster%2FCONTRIBUTING.md&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982845173&sdata=9lmxgxO8gS8NAyxPDilwgn1LeBqgoBABEh4h7zoyHeE%3D&reserved=0>>.

* #3097: Is CONNECTION_CLOSE ACK-eliciting?
   The proposal is <https://github.com/quicwg/base-drafts/issues/3097<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fissues%2F3097&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982845173&sdata=ywG1Tb9q4fUHIwRDOnRFmab1bY%2B4uCwQesA111%2FXAWA%3D&reserved=0>>

* #3085: Stateless reset detection should be datagram-based
   The proposal is <https://github.com/quicwg/base-drafts/pull/2993<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F2993&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982855130&sdata=PGbEbWSbA%2BsMiaN8PNv96Bt1fF%2Fj%2F6wSplJuBRml7vc%3D&reserved=0>>

* #3054: Label for key updates
   The proposal is <https://github.com/quicwg/base-drafts/pull/3050<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3050&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982855130&sdata=b5WqdtRz1t%2FKfH4QWudxqnuKET9CL2p1YuTQ5p4rhUQ%3D&reserved=0>>

* #3046: Handling of Retire Prior To field
   The proposal is <https://github.com/quicwg/base-drafts/pull/3096<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3096&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982855130&sdata=YRIcwVV9sR9Yn%2FKPqw%2Fv9tds07rU4r9awQKKtHyg%2FKs%3D&reserved=0>>

* #3037: Require peers to check if RETIRE_CONNECTION_ID sequence number is valid
   The proposal is <https://github.com/quicwg/base-drafts/pull/3036<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3036&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982865085&sdata=0yIMdKA5L5GVX%2Bn0fwd57VX%2FPeQteXWo7rG3QKUVb9o%3D&reserved=0>>

* #3027: Codes for frame encoding errors
   The proposal is <https://github.com/quicwg/base-drafts/pull/3042<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3042&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982865085&sdata=xAFv60554CFXTY2skAcJbLdj%2BxnhvJJI9FsrQUIG9Ws%3D&reserved=0>>

* #2928: Lift single-packet ClientHello requirement?
   The proposal is <https://github.com/quicwg/base-drafts/pull/3045<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3045&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982865085&sdata=3oZHg77Z%2Fk0e8E8O9rZdlStcvN%2BhqEd3e3kEXPyNW9s%3D&reserved=0>>

* #2863: unrecoverable loss pattern leads to deadlock
   The proposal is <https://github.com/quicwg/base-drafts/pull/3121<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F3121&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982875044&sdata=2Hcsmy6VVVeZwWTRCcPm%2F9NfjXGgYctNw%2FktWswy4e4%3D&reserved=0>>

I feel sorry but I object to the proposed approach. As I have pointed out in my comments in #2863, not discarding the Handshake key forever breaks our assumption that no Handshake packets will be sent or received once the handshake is confirmed. We missed that there are these side effects when we rush to the conclusion at the end of the interim.

Specifically, we'd be required to define how Handshake packets deal with migration (does the path for Handshake packets change? If it does, how do we fill in the SCID field of the Handshake packet? If it does not, until when do the endpoints need to retain the original path?).

I think we should fix the deadlock, rather than changing other parts of the protocol that currently depend on the Handshake key being discarded at some point.

* #2823: Do Initial secrets change after Retry packet?
   The proposal is <https://github.com/quicwg/base-drafts/pull/2870<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F2870&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982875044&sdata=4boCzjNG9GOOIhdxKzuJVMvxK4UANTkFApQsSVexoaU%3D&reserved=0>>

* #2152: Why does stateless reset have to be checked after MAC failure
   The proposal is <https://github.com/quicwg/base-drafts/pull/2993<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F2993&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982884999&sdata=4t3wu7IoW44SjJNkxJSKJZ8Lo23YtF%2FCM4%2FE%2B%2FW89FY%3D&reserved=0>>

--
Mark Nottingham   https://www.mnot.net/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mnot.net%2F&data=02%7C01%7Cnibanks%40microsoft.com%7C792ac24eb30d48387a8508d7568f93de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637073046982884999&sdata=YlH%2BenfkuPhwh0GMcRkRlpqxG%2BChyG9bGtYQ9huP6wo%3D&reserved=0>

--
Kazuho Oku

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 22 Oct 2019 13:38:23 +1100

> something like RETIRE_KEY.

I would prefer to think of this proposed resolution as a temporary one.  I don't think that we agreed to keep the handshake keys indefinitely, only that we would use that option as a fallback position until we found a better solution.

On that basis, I think that it would be best if we open a new issue that says "Handshake keys can't ever be dropped".  

We might still conclude not to address that issue, but the important thing is to ensure that any solution works properly.

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 22 Oct 2019 13:57:38 +0900

2019年10月22日(火) 11:38 Martin Thomson <[email-masked]>:

> I would prefer to think of this proposed resolution as a temporary one.  I
> don't think that we agreed to keep the handshake keys indefinitely, only
> that we would use that option as a fallback position until we found a
> better solution.
>

I might point out that #3121 is not proposal-ready as a way to resolve
#2863. That is because it does not define how to send and receive Handshake
packets until or after migration happens. There would be a deadlock unless
both endpoints agree on how that should be done (e.g., how to select SCID,
whether the path used for Handshake packets migrates too).

Without that being clarified, can we say that we are ready for a
consensus call?

>
> On that basis, I think that it would be best if we open a new issue that
> says "Handshake keys can't ever be dropped".
>
> We might still conclude not to address that issue, but the important thing
> is to ensure that any solution works properly.
>
>

-- 
Kazuho Oku

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Tue, 22 Oct 2019 05:14:31 +0000

Without a way to move to a single PN space I find this a deal breaker. I might do a custom version of the protocol.

________________________________
Fra: QUIC <[email-masked]> på vegne af Kazuho Oku <[email-masked]>
Sendt: tirsdag, oktober 22, 2019 6:58 AM
Til: Martin Thomson
Cc: IETF QUIC WG
Emne: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

2019年10月22日(火) 11:38 Martin Thomson <[email-masked]<mailto:[email-masked]>>:

I might point out that #3121 is not proposal-ready as a way to resolve #2863. That is because it does not define how to send and receive Handshake packets until or after migration happens. There would be a deadlock unless both endpoints agree on how that should be done (e.g., how to select SCID, whether the path used for Handshake packets migrates too).

Without that being clarified, can we say that we are ready for a consensus call?

--
Kazuho Oku

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 22 Oct 2019 16:50:53 +0900

I agree with Kazuho here. The issue of migration was not one we had
considered. I was going through this with Kazuho, and we may have
identified yet another issue with handshake retransmissions and migration,
which is present in the current spec.

Sadly, I don't think we can call this issue done.

On Tue, Oct 22, 2019 at 2:14 PM Mikkel Fahnøe Jørgensen <[email-masked]>
wrote:

> Without a way to move to a single PN space I find this a deal breaker. I
> might do a custom version of the protocol.
>
>
> ------------------------------
> *Fra:* QUIC <[email-masked]> på vegne af Kazuho Oku <
> [email-masked]>
> *Sendt:* tirsdag, oktober 22, 2019 6:58 AM
> *Til:* Martin Thomson
> *Cc:* IETF QUIC WG
> *Emne:* Re: Consensus Calls for Transport/TLS issues, post-Cupertino
>
>
>
>> I would prefer to think of this proposed resolution as a temporary one.
>> I don't think that we agreed to keep the handshake keys indefinitely, only
>> We might still conclude not to address that issue, but the important
>> thing is to ensure that any solution works properly.
>>
>>
>
> --
> Kazuho Oku
>

---

From: David Schinazi <[email-masked]>
Date: Tue, 22 Oct 2019 13:06:33 -0700

+1

I think the text we have in the spec today is better than the proposal to
never discard the handshake keys.
My preference would be to spend the time to fix the issue, and not add a
temporary workaround for now.

On Tue, Oct 22, 2019 at 12:51 AM Jana Iyengar <[email-masked]> wrote:

> On Tue, Oct 22, 2019 at 2:14 PM Mikkel Fahnøe Jørgensen <
> [email-masked]> wrote:
>

---

From: Mike Bishop <[email-masked]>
Date: Wed, 23 Oct 2019 18:53:31 +0000

I think we’ve amply demonstrated that implicit signals don’t work.  I’d like to see an explicit confirmation of handshake completion that happens at 1-RTT.

From: QUIC <[email-masked]> On Behalf Of David Schinazi
Sent: Tuesday, October 22, 2019 4:07 PM
To: Jana Iyengar <[email-masked]>
Cc: Mikkel Fahnøe Jørgensen <[email-masked]>; IETF QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>; Kazuho Oku <[email-masked]>
Subject: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

+1

I think the text we have in the spec today is better than the proposal to never discard the handshake keys.
My preference would be to spend the time to fix the issue, and not add a temporary workaround for now.

On Tue, Oct 22, 2019 at 12:51 AM Jana Iyengar <[email-masked]<mailto:[email-masked]>> wrote:
I agree with Kazuho here. The issue of migration was not one we had considered. I was going through this with Kazuho, and we may have identified yet another issue with handshake retransmissions and migration, which is present in the current spec.

On Tue, Oct 22, 2019 at 2:14 PM Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>> wrote:
Without a way to move to a single PN space I find this a deal breaker. I might do a custom version of the protocol.

________________________________
Fra: QUIC <[email-masked]<mailto:[email-masked]>> på vegne af Kazuho Oku <[email-masked]<mailto:[email-masked]>>
Sendt: tirsdag, oktober 22, 2019 6:58 AM
Til: Martin Thomson
Cc: IETF QUIC WG
Emne: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

--
Kazuho Oku

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 24 Oct 2019 11:39:44 +0900

Yup. We need to get this over with in the dumbest way possible.

On Thu, Oct 24, 2019 at 3:53 AM Mike Bishop <[email-masked]> wrote:

> I think we’ve amply demonstrated that implicit signals don’t work.  I’d
> like to see an explicit confirmation of handshake completion that happens
> at 1-RTT.
>
>
>
> *From:* QUIC <[email-masked]> *On Behalf Of * David Schinazi
> *Sent:* Tuesday, October 22, 2019 4:07 PM
> *To:* Jana Iyengar <[email-masked]>
> *Cc:* Mikkel Fahnøe Jørgensen <[email-masked]>; IETF QUIC WG <
> [email-masked]>; Martin Thomson <[email-masked]>; Kazuho Oku <
> [email-masked]>
> *Subject:* Re: Consensus Calls for Transport/TLS issues, post-Cupertino
>
>
>
> +1
>
>
>

---

From: Martin Duke <[email-masked]>
Date: Fri, 25 Oct 2019 09:36:05 -0700

I agree with the dissenters. The current spec works in the HTTP3 use case,
though I think we should fix it. Keeping keys forever is unsatisfactory for
several reasons. We should use an explicit signal, and there is no reason
to kick the can down the road. Let's just leave #2863 open till we resolve
it.

On Wed, Oct 23, 2019 at 7:40 PM Jana Iyengar <[email-masked]> wrote:

>> Sadly, I don't think we can call this issue done..
>>
>>
>>

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 27 Oct 2019 16:01:43 -0700

I think we're clearly going to need to spend some time on this. I don't
think the spec is satisfactory as-is: we should be designing a transport
that works for all use cases, not just H3. That said, I also don't agree
that we need an additional explicit signal. We have one, it's called "ACK".
We should figure out how to make that work.
-Ekr

On Fri, Oct 25, 2019 at 9:36 AM Martin Duke <[email-masked]> wrote:

>>> Sadly, I don't think we can call this issue done...
>>>
>>>
>>>

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 27 Oct 2019 16:02:39 -0700

Note that we also can't close 2741 until we resolve 2863.

-Ekr

On Sun, Oct 27, 2019 at 4:01 PM Eric Rescorla <[email-masked]> wrote:

>> I agree with the dissenters. The current spec works in the HTTP3 use
>> case, though I think we should fix it. Keeping keys forever is
>> unsatisfactory for several reasons. We should use an explicit signal, and
>> there is no reason to kick the can down the road. Let's just leave #2863
>> open till we resolve it.
>>
>>>> Without a way to move to a single PN space I find this a deal breaker.
>>>> I might do a custom version of the protocol.
>>>>
>>>>
>>>>
>>>>
>>>> I would prefer to think of this proposed resolution as a temporary
>>>> one.  I don't think that we agreed to keep the handshake keys indefinitely,
>>>> only that we would use that option as a fallback position until we found a
>>>> better solution.
>>>>
>>>>
>>>>
>>>> On that basis, I think that it would be best if we open a new issue
>>>> that says "Handshake keys can't ever be dropped".
>>>>

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 28 Oct 2019 13:46:51 +1100

On Mon, Oct 28, 2019, at 10:01, Eric Rescorla wrote:
> think the spec is satisfactory as-is: we should be designing a 
> transport that works for all use cases, not just H3. That said, I also 
> don't agree that we need an additional explicit signal. We have one, 
> it's called "ACK". We should figure out how to make that work.

I think that we've established that ACK doesn't work without something to push it along.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 28 Oct 2019 16:34:16 -0700

> I think that we've established that ACK doesn't work without something to

No, I don't agree with that assessment.

-Ekr

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 29 Oct 2019 14:31:54 +1100

> No, I don't agree with that assessment.

Well, I'd like some proof.  I shared what I believe to be proof earlier.  Happy to walk through it again though.

---

From: Mike Bishop <[email-masked]>
Date: Tue, 29 Oct 2019 18:26:39 +0000

Let me make sure I’ve got things in the right order here: things “just work” if both client and server are actively sending 1-RTT data.  The server knows the handshake is complete when it gets the Client Finished; the problem is that the server doesn’t know.

Therefore, ACK works iff we know that the client sent something ack-eliciting in 1-RTT.  So one solution is to require clients to send something – anything, really – in 1-RTT immediately after the Finished.  When any 1-RTT packet is ACK’d, the client knows the server has confirmed the handshake.  A client that wanted to be cautious could simply send an immediate PING, or PATH_CHALLENGE at 1-RTT along with the Finished, and they’d get their ACK.  (This only works if the server is doing the right thing and not reading 1-RTT packets until it sees the Finished, however.)  Requiring the client to send HANDSHAKE_DONE would have the same effect, but require it instead of permit it.

Alternatively, the server could send HANDSHAKE_DONE, explicitly declaring that it has received the Finished.  Receipt of the frame is all the client needs, but the server needs the ACK to know when to stop sending it.

From: Eric Rescorla <[email-masked]>
Sent: Monday, October 28, 2019 7:34 PM
To: Martin Thomson <[email-masked]>
Cc: Martin Duke <[email-masked]>; Jana Iyengar <[email-masked]>; Mike Bishop <[email-masked]>; David Schinazi <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; IETF QUIC WG <[email-masked]>; Kazuho Oku <[email-masked]>
Subject: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

On Sun, Oct 27, 2019 at 7:47 PM Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

-Ekr

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 30 Oct 2019 10:45:59 +1100

On Wed, Oct 30, 2019, at 05:26, Mike Bishop wrote:
> Therefore, ACK works iff we *know* that the client sent something 
> ack-eliciting in 1-RTT. 

I think that you have to make it symmetric, not just "client sends".  Because we don't have implicit acknowledgement for Handshake packets from the server, the server can enter the same state that the client does in Marten's deadlock scenario.  In other words, both need to send something ack-eliciting.  And it's not just anything at any time, it has to be anything within some bounded time.  The upper bound is probably high, but it is at least before the Handshake packets consume all the available congestion window (which might be a long time, but I haven't worked out how long that could take).

> So one solution is to require clients to send 

...and servers :)

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 29 Oct 2019 16:58:52 -0700

> Happy to walk through it again though.

Yeah, I think that would help. I've gotten a little lost on the list, so
maybe you can re-point it out to me.

-Ekr

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 30 Oct 2019 16:02:28 +1100

On Wed, Oct 30, 2019, at 10:58, Eric Rescorla wrote:
> Yeah, I think that would help. I've gotten a little lost on the list, 
> so maybe you can re-point it out to me.

Let me try restating it more thoroughly:

The problem that Marten identified is a deadlock because one side discards keys at a time that might be significantly in advance of the other.  If the endpoint that retains keys still hasn't received acknowledgments, it enters a deadlock state if it exhausts the congestion window by sending Handshake packets.

(As an aside, David suggested that we might avoid deadlock if the congestion controller didn't create a dependency cycle between 1-RTT and Handshake, but I didn't detect any inclination to do that, nor am I completely confident that attempting to do so would guarantee success.  There's also the problem that Kazuho and Jana identified, which observes that a loose upper bound on convergence time was undesirable if we want to enable migration.)

Both peers can enter this potential deadlock state in different ways, so we need a symmetric solution.  I can draw up what this looks like in converse, but just imagine that the client sends and loses a Handshake ACK that is separate from the Finished.  That side of this could be addressed with implicit acknowledgment for the server, but I don't think we want to go there.

Any signal that is based purely on "organic" traffic (that is stuff that is sent by the application) can produce an asymmetric state where one side receives the signal and the other does not.  If both peers stop sending packets at that point, which "organic" traffic would have to permit, then the keys will be gone on one side and not the other.

In general, frames from an endpoint don't result in any mandatory transport-level action other than an ACK frame.  ACK frames don't elicit acknowledgment and aren't sent again.  PATH_CHALLENGE is the other way to get a transport-level forced response, but that is similar in that PATH_RESPONSE also doesn't require retransmision.  An endpoint that sends a packet can decide not to send another packet.  Thus, any implicit signal we create can only guarantee one-way propagation, which is firmly in two generals territory as Christian points out.

A PATH_CHALLENGE is almost right for this purpose, because it initiates a three-way exchange, but the lack of requirements for retransmission makes it difficult to guarantee that the process terminates.  And that probably requires overloading semantics for the frames in uncomfortable ways.  Forcing either endpoint to repeatedly initiate path validation would be worse than HANDSHAKE_DONE, especially when you consider that the path is already validated by the handshake.

Thus, while we might be able to define a process where each peer can satisfy themselves that their peer is "done", we don't have an "organic" process that we can attach to for roughly synchronizing the times that endpoints reach that conclusion.

---

From: Watson Ladd <[email-masked]>
Date: Tue, 29 Oct 2019 22:08:48 -0700

> Thus, while we might be able to define a process where each peer can satisfy themselves that their peer is "done", we don't have an "organic" process that we can attach to for roughly synchronizing…

Maybe it's worth writing down the state machine formally and trying to
use a model checker to show that there are no deadlocking pathways.
It's certainly confusing enough for me to have no idea.

-- 
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 30 Oct 2019 17:51:49 +1100

> It's certainly confusing enough for me to have no idea.

You aren't the first one to point this out.  I think that the primary concern was lack of time and probably also lack of expertise.  There are people looking into finding/funding work on validation of our handshake.

---

From: Mike Bishop <[email-masked]>
Date: Wed, 30 Oct 2019 15:45:45 +0000

And this is the piece I'm not getting, I think.

The minimum requirement for the server to know the client got the whole handshake is to receive the Client Finished; receipt of any 1-RTT traffic from the client demonstrates they derived the correct keys.  The server can drop Handshake keys at that point.  However, the server can no longer retransmit the ACK or even know if it was delivered, so the client needs a reliable sync point based on 1-RTT traffic to know that the server received the Client Finished.

Since the server can send 1-RTT packets before the handshake completes, receipt of 1-RTT packets isn't sufficient.  Since the 0-RTT and 1-RTT packets share a space, receiving an ACK in a 1-RTT packet isn't sufficient.  The client knows that the server has seen the whole handshake once it receives an ACK of a 1-RTT packet.  As soon as any packet sent in 1-RTT is ACK'd, the client can drop handshake keys.

As you explain in the other fork, organic traffic isn't sufficient, so we have to force something into 1-RTT to ensure these signals happen.  If the client sends HANDSHAKE_DONE in 1-RTT, probably coalesced with Client Finished, that's an ack-eliciting 1-RTT packet.  The server will ACK it; if the ACK gets lost, the client will PTO its 1-RTT packet and the server will ack *that*, and so on.  Eventually the client gets an ACK of a 1-RTT packet, which is what it needs.  It drops keys.

If the server sends HANDSHAKE_DONE upon receipt of the Finished, that is a more direct signal, and the client can drop keys upon receipt of that frame.  If the ACK of the server's HANDSHAKE_DONE gets lost, the server will PTO it, and the client will ACK, and so on. But those things aren't really necessary -- the client and server have both already gotten their signal.  I can see the appeal of having the server send it, so that we're not driving off of ACKs of particular packets.

The only reason I see for both sides to send it is distaste for implicit acknowledgements, trying to avoid letting the Client Finished be what tells the server that all its Handshake data clearly made it.  Is this just preference for one solution over another, or is there a technical reason I'm missing?

-----Original Message-----
From: QUIC <[email-masked]> On Behalf Of Martin Thomson
Sent: Tuesday, October 29, 2019 7:46 PM
To: [email-masked]
Subject: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

> So one solution is to require clients to send

....and servers :)

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 30 Oct 2019 12:07:35 -0400

I don’t see a need for HANDSHAKE_DONE in both directions although it won’t
hurt.

An argument for symmetric HANDSHAKE_DONE is that it decouples the handshake
transition from the handshake logic which makes it easier to replace TLS
with something simpler, for example in a QUIC version for constrained
devices.

On 30 October 2019 at 16.46.10, Mike Bishop ([email-masked]) wrote:

The minimum requirement for the server to know the client got the whole
handshake is to receive the Client Finished; receipt of any 1-RTT traffic
from the client demonstrates they derived the correct keys. The server can
drop Handshake keys at that point. However, the server can no longer
retransmit the ACK or even know if it was delivered, so the client needs a
reliable sync point based on 1-RTT traffic to know that the server received
the Client Finished.

Since the server can send 1-RTT packets before the handshake completes,
receipt of 1-RTT packets isn't sufficient. Since the 0-RTT and 1-RTT
packets share a space, receiving an ACK in a 1-RTT packet isn't sufficient.
The client knows that the server has seen the whole handshake once it
receives an ACK of a 1-RTT packet. As soon as any packet sent in 1-RTT is
ACK'd, the client can drop handshake keys.

As you explain in the other fork, organic traffic isn't sufficient, so we
have to force something into 1-RTT to ensure these signals happen. If the
client sends HANDSHAKE_DONE in 1-RTT, probably coalesced with Client
Finished, that's an ack-eliciting 1-RTT packet. The server will ACK it; if
the ACK gets lost, the client will PTO its 1-RTT packet and the server will
ack *that*, and so on. Eventually the client gets an ACK of a 1-RTT packet,
which is what it needs. It drops keys.

If the server sends HANDSHAKE_DONE upon receipt of the Finished, that is a
more direct signal, and the client can drop keys upon receipt of that
frame.. If the ACK of the server's HANDSHAKE_DONE gets lost, the server
will PTO it, and the client will ACK, and so on. But those things aren't
really necessary -- the client and server have both already gotten their
signal. I can see the appeal of having the server send it, so that we're
not driving off of ACKs of particular packets.

The only reason I see for both sides to send it is distaste for implicit
acknowledgements, trying to avoid letting the Client Finished be what tells
the server that all its Handshake data clearly made it. Is this just
preference for one solution over another, or is there a technical reason
I'm missing?

-----Original Message-----
From: QUIC <[email-masked]> On Behalf Of Martin Thomson
Sent: Tuesday, October 29, 2019 7:46 PM
To: [email-masked]
Subject: Re: Consensus Calls for Transport/TLS issues, post-Cupertino

> Therefore, ACK works iff we *know* that the client sent something

I think that you have to make it symmetric, not just "client sends".
Because we don't have implicit acknowledgement for Handshake packets from
the server, the server can enter the same state that the client does in
Marten's deadlock scenario. In other words, both need to send something
ack-eliciting. And it's not just anything at any time, it has to be
anything within some bounded time. The upper bound is probably high, but it
is at least before the Handshake packets consume all the available
congestion window (which might be a long time, but I haven't worked out how
long that could take).

> So one solution is to require clients to send

.....and servers :)

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 30 Oct 2019 14:00:44 -0700

On Tue, Oct 29, 2019 at 10:02 PM Martin Thomson <[email-masked]> wrote:

> > >  Well, I'd like some proof. I shared what I believe to be proof
> earlier. Happy to walk through it again though.
> >
> The problem that Marten identified is a deadlock because one side discards
> keys at a time that might be significantly in advance of the other.  If the
> endpoint that retains keys still hasn't received acknowledgments, it enters
> a deadlock state if it exhausts the congestion window by sending Handshake
> packets.
>

I'm going to stop here, because it seems that you're assuming we don't just
keep the keys indefinitely. How does that lead to deadlock?

-Ekr

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 30 Oct 2019 14:06:56 -0700

The problem is that having keys indefinitely leads to various problems. If
the goal is to decide when the handshake is complete at both ends in finite
time with high probability, then it appears that an explicit signal is
required because you cannot rely on random (organic) 1-RTT traffic to drive
this process.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 30 Oct 2019 14:21:19 -0700

On Wed, Oct 30, 2019 at 2:06 PM Mikkel Fahnøe Jørgensen <[email-masked]>
wrote:

>
> I'm going to stop here, because it seems that you're assuming we don't
> just keep the keys indefinitely. How does that lead to deadlock?
>
>
> The problem is that having keys indefinitely leads to various problems.
>
I've heard that asserted but I'm not so sure.

> If the goal is to decide when the handshake is complete at both ends in
> finite time with high probability, then it appears that an explicit signal
> is required because you cannot rely on random (organic) 1-RTT traffic to
> drive this process.
>
Well, it's not clear to me that one needs to know this, actually. I agree
we've used this as a proxy for various things, but it's not actually clear
to me that we need to, so we should re-examine those cases to see what the
real preconditions are.

-Ekr

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 30 Oct 2019 14:33:17 -0700

For one thing, I would be sad if an optimised implementation needs to
handle multiple packet number spaces. That complicates implementations
because you need to think performance through all layers. Of course you can
just suck it up an deal with it. Likewise, you can probably find a way
around path migration.

By compared to sending a single 1-byte HANDSHAKE_DONE frame, why would you
do that? To marginally simplify the handshake?

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 30 Oct 2019 14:48:46 -0700

> do that? To marginally simplify the handshake?
This isn't an argument that keeping the keys around doesn't work, but
rather that you think it's the wrong engineering tradeoff. It's possible
you're right about that, but I'd like to start by establishing the agreed
upon facts of what will and will not work before we debate engineering
tradeoffs.

-Ekr

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 31 Oct 2019 11:54:06 +1100

On Thu, Oct 31, 2019, at 08:21, Eric Rescorla wrote:
> On Wed, Oct 30, 2019 at 2:06 PM Mikkel Fahnøe Jørgensen 
> <[email-masked]> wrote:
> >> I'm going to stop here, because it seems that you're assuming we don't just keep the keys indefinitely. How does that lead to deadlock?
> > 
> 

The problems I'm aware of are largely edifices of our own construction.

The primary one I am aware of is that we don't know what to do about sending Handshake packets after a migration.  This is fixable, because it is a definitional problem.  We haven't defined what to do.  The problem is, I think just limited to the use of Source Connection ID on long header packets.  I personally would prefer not to have to fix this one.

If you don't continue sending Handshake packets, that has the same effect as discarding keys for the side that is being starved of acknowledgments.

We could also build, as Mike alludes to, an implicit acknowledgment.  We currently have one for the Initial->Handshake transition and we could build another.  That only reduces this from a symmetric issue to an asymmetric one unless we address the existing problem.

If we were able to reduce this to just a client-side problem, then we might also say that clients can't migrate prior to verifying that all Handshake data was transmitted to as to avoid any need for problems.  Then migration becomes proof positive that the client is done transmitting and that can be the signal.

Of course, this is all getting increasingly byzantine.

---

From: Christian Huitema <[email-masked]>
Date: Wed, 30 Oct 2019 18:32:27 -0700

> If you don't continue sending Handshake packets, that has the same effect as discarding keys for the side that is being starved of acknowledgments.

There is the half-way step of only sending acknowledgements, only upon
receipt of new Handshake packets.

> We could also build, as Mike alludes to, an implicit acknowledgment.  We currently have one for the Initial->Handshake transition and we could build another.  That only reduces this from a symmetri…

I would assume that implementations, in practice, will consider implicit
acknowledgements. This is fairly obvious on the server side: receiving
the "handshake complete" from the stack means that all required
handshake packets have been received by both parties. The server knows
that there is no point in re-transmitting any of the previous handshake
packets. It can implement the "implicit ack" logic without breaking
anything on the server side. I assume that implementations will just do
that, no matter what the Quic spec says, because it makes the server
stack robust to silly scenario in which the client's acks of handshake
packets get lost. Which means that the Quic spec might just as well say it.

So the problem really is asymmetric. The client is fine, because it
eventually receives an ACK of its final handshake packet can can
consider the handshake confirmed at that point. But the server never
knows whether the ack of the final handshake was properly received.

> If we were able to reduce this to just a client-side problem, then we might also say that clients can't migrate prior to verifying that all Handshake data was transmitted to as to avoid any need fo…

Migration is only initiated by the client in V1, so waiting for the last
handshake ACK on the client side is good enough. But key rotation could
be initiated by the server, and doing that before handshake confirmed is
a bit problematic. Which means that the server needs to know whether the
client's handshake is confirmed, which is where the Handshake Done
signal becomes useful. My preference would be for the server to send
that signal and repeat it until it is acknowledged by the client. When
that ACK is received, the server knows that the handshake is confirmed.

-- Christian Huitema

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 31 Oct 2019 12:44:30 +1100

On Thu, Oct 31, 2019, at 12:32, Christian Huitema wrote:
> a bit problematic. 

I don't see how this is problematic.  Handshake packets use Handshake keys, so they aren't updated.

---

From: Christian Huitema <[email-masked]>
Date: Wed, 30 Oct 2019 19:41:57 -0700

> I don't see how this is problematic.  Handshake packets use Handshake keys, so they aren't updated.

Yes, that should work. The client could also consider the
server-initiated key rotation as an implicit acknowledgement of the
Handshake packets, since the server is only supposed to initiate a
rotation after Handshake Confirmed. The client can delete the Handshake
Key at that point.

In fact, If the server successfully sees the client rotating the keys,
it can induce that the client has received an implicit ack of all the
handshake packets, and it too can delete its Handshake Keys. So if
either side initiates a key rotation and the key rotation succeeds, both
sides can jettison the Handshake keys -- provided the server only
initiates after receiving the Client Finished message, and the client
only initiates after receiving an ACK of that message.

So we have a solution that works if the server if willing to keep the
Handshake key forever, and allows the server to jettison the Handshake
keys if it is willing to rotate the 1RTT key.

-- Christian Huitema

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 31 Oct 2019 13:52:34 +1100

With the exception of #2863 (where we'll continue discussion), consensus declared.

Cheers,

> On 22 Oct 2019, at 11:42 am, Mark Nottingham <[email-masked]> wrote:
>