Subject: Re: Second Implementation Draft Guidelines

From: Jana Iyengar <[email-masked]>
Date: Sat, 8 Jul 2017 09:45:49 -0700

I've been thinking about this, and I'm starting to think that we should
cover more ground in the second implementation draft.

I'm hearing about increasing deployments of gQUIC, largely due to market
pressures. The availability of the Chromium implementation makes it
particularly easy for folks to deploy QUIC with that code. I think we need
to move with some urgency, even if we don't change everything about QUIC to
make it perfect, so that we can start getting IETF QUIC deployments out
there. Specifically, I think we should:
1. work out the wire-visible invariants and finalize all of those for the
second impl draft. We know that there are some middleboxes that already
have classifiers for gQUIC, and we need to move quickly and push IETF-QUIC
so we can test that IETF-QUIC is deployable. I fear that the longer we
take, the more widespread gQUIC ossification will be.
2. allow impls to make serious progress towards a basic HTTP mapping over
QUIC. We can punt on header compression (QPACK/QCRAM), but perhaps test a
basic HTTP request-response over QUIC. We can still punt
performance-oriented things such as full loss recovery and congestion
control to later. This forces us to try and finalize the HTTP mapping
details, which is a good thing, IMO.

On Thu, Jun 22, 2017 at 10:07 AM, Lucas Pardue <[email-masked]>
wrote:

> I don’t feel I came away from the Paris Interim with a clear idea of the
> approach to the second implementation draft. There was a lot of discussion
> around whether bugfixes in new I-D drafts (that address issues identified
> in the first implementation draft) would contribute to a first.1
> implementation draft (all the way up to first.N) or if they would be put
> into the second.
>
>
>
> Side note: Personally, I don’t particularly like the terminology of first,
> second etc for the reason that it gets hard to rev/iterate on a particular
> one. However, perhaps it was purposely chosen to disambiguate from the I-D
> versions?
>
>
>
> The strawman suggests to me that the direction is to lump both fixes and
> new features into the second implementation draft. I don’t necessarily have
> an argument against that but think it would help to clearly identify where
> things have been fixed, changed completely, been overtaken by other changes
> etc. Understandably it is too early to do that at this stage.
>
>
>
> Finally, being super picky, the second implementation doesn’t seem to
> cover non-critical shortcomings
>
>
>
> Regards
>
> Lucas
>
>
>
> *From:* QUIC [mailto:[email-masked]] *On Behalf Of *Jana Iyengar
> *Sent:* 21 June 2017 22:23
> *To:* Martin Duke <[email-masked]>
> *Cc:* IETF QUIC WG <[email-masked]>
> *Subject:* Re: Second Implementation Draft Guidelines
>
>
>
> This is a great start, thanks for getting it going.
>
> Just one thought: You may want to explicitly "include" fixed-time
> RTO-recovery. It appears right now as a side note in what's not included.
>
>
>
> On Wed, Jun 21, 2017 at 2:11 PM, Martin Duke <[email-masked]>
> wrote:
>
> All,
>
>
>
> As promised, I've created a set of guidelines for what should be in the
> Second Implementation draft. If deadlines hold, I believe we will actually
> start implementing this draft after Seattle in October.
>
>
>
> Once we've reached reasonable consensus, this should serve as a basis for
> prioritizing issues to resolve through Seattle.
>
>
>
> The current version is very much a starting point for discussion. I've
> presented some items I think are important to include, and others that we
> could bring in depending on our appetite for work. The final version will
> simply have "Must Include" and "should not include".
>
>
> <http://goog_314636520>
>
> https://github.com/quicwg/base-drafts/wiki/Second-Implementation-Draft
>
>
>
> I look forward to your comments.
>
>
>
> Martin
>
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 9 Jul 2017 08:12:00 +0200

Hi Jana,

> details, which is a good thing, IMO.

Just be *very* careful not to fall again into the trap that we've met with
H2 where existing deployments prevented some final bits of the protocol
from being fixed. Implementations are very important to get feedback on
the protocol and the related issues but at some point they start to lead
the development by deciding that it's "good enough" to be shipped. This
can very likely be addressed by showing frequent progress on certain
points that are possibly not currently being addressed in implementations
to make implementers consider that their deployment is only an acceptable
temporary solution while waiting for a much better final one.

Just my two cents,
Willy

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 10 Jul 2017 09:39:01 +0900

2017-07-09 1:45 GMT+09:00 Jana Iyengar <[email-masked]>:
> second impl draft. We know that there are some middleboxes that already have
> classifiers for gQUIC, and we need to move quickly and push IETF-QUIC so we
> can test that IETF-QUIC is deployable. I fear that the longer we take, the
> more widespread gQUIC ossification will be.

I agree with Jana.

If we can have some basic HTTP mapping (it can be as basic as using
HTTP/1.0 over each stream), we can use that to test how the IETF
version of QUIC performs well in the field, by comparing its
performance to HTTP over TCP.

>> around whether bugfixes in new I-D drafts (that address issues identified in
>> the first implementation draft) would contribute to a first.1 implementation
>> draft (all the way up to first.N) or if they would be put into the second.
>>
>>
>>
>> From: QUIC [mailto:[email-masked]] On Behalf Of Jana Iyengar
>> Sent: 21 June 2017 22:23
>> To: Martin Duke <[email-masked]>
>> Cc: IETF QUIC WG <[email-masked]>
>> Subject: Re: Second Implementation Draft Guidelines
>>
>>
>>

-- 
Kazuho Oku

---

From: Ryan Hamilton <[email-masked]>
Date: Sun, 9 Jul 2017 20:28:30 -0700

On Sun, Jul 9, 2017 at 5:39 PM, Kazuho Oku <[email-masked]> wrote:

> have

Interesting idea. One challenge with performance analysis is that it'll be
a bit of an apples to oranges comparison. QUIC will be doing HTTP/1
(without header compression) against HTTP/2 (with header compression) or
HTTP/1.1 (over multiple connections).

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 10 Jul 2017 16:32:14 +0900

2017-07-10 12:28 GMT+09:00 Ryan Hamilton <[email-masked]>:
> Interesting idea. One challenge with performance analysis is that it'll be a
> bit of an apples to oranges comparison. QUIC will be doing HTTP/1 (without
> header compression) against HTTP/2 (with header compression) or HTTP/1.1
> (over multiple connections).

Agreed.

Though I might argue that collecting metrics of a QUIC implementation
without header compression could be useful. We can use that as a
baseline when we formalize QPACK / QCRAM.

-- 
Kazuho Oku

---

From: Ian Swett <[email-masked]>
Date: Mon, 10 Jul 2017 06:33:53 -0400

There are two practical options to mitigate performance issues on opposite
ends of the spectrum, and both of which I expect to perform at least
slightly better than H2 over TCP for most applications.

1) Run HTTP/2 over a single QUIC stream.  This should be easy for most
implementers, but incurs maximum HOL blocking.
2) Use the HPACK static dictionary to get some of the header compression
benefits.  I think this provides a better baseline to evaluate QPACK/QCRAM
from.

On a high level, I support adding some sort of HTTP application mapping to
the second draft, with the knowledge it is likely to change a large
amount.  GQUIC has already changed it's HTTP mapping at least twice that
I'm aware of, and it has some challenges, but it's fairly doable, even when
supporting both for long periods of time.

On Mon, Jul 10, 2017 at 3:32 AM, Kazuho Oku <[email-masked]> wrote:

> >> > to move with some urgency, even if we don't change everything about
> QUIC
> >> > to
> >> > classifiers for gQUIC, and we need to move quickly and push IETF-QUIC
> so
> >> > we
> >> > QUIC. We can punt on header compression (QPACK/QCRAM), but perhaps
> test
> >> > a

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Mon, 10 Jul 2017 15:12:01 +0200

> On 10 Jul 2017, at 12:33, Ian Swett <[email-masked]> wrote:
> 
> There are two practical options to mitigate performance issues on opposite ends of the spectrum, and both of which I expect to perform at least slightly better than H2 over TCP for most applications.
> 
> 1) Run HTTP/2 over a single QUIC stream.  This should be easy for most implementers, but incurs maximum HOL blocking.
> 2) Use the HPACK static dictionary to get some of the header compression benefits.  I think this provides a better baseline to evaluate QPACK/QCRAM from.
> 
> On a high level, I support adding some sort of HTTP application mapping to the second draft, with the knowledge it is likely to change a large amount.

This seems like a good target, as long as it's possible to ensure it's blindingly obvious to everyone that it is likely to change a large amount.

We have a bit of a paradox here. In order to keep people from shipping 2nd impl it needs to be functionally limited to the point that it's obviously silly to ship. But it can only be useful as a test of the features in the spec that it implements, which argues against an obviously silly profile.

A useful question to ask at this point: how many separate implementation drafts do we target before WGLC?

Cheers,

Brian

>  GQUIC has already changed it's HTTP mapping at least twice that I'm aware of, and it has some challenges, but it's fairly doable, even when supporting both for long periods of time.
>

---

From: Martin Duke <[email-masked]>
Date: Mon, 10 Jul 2017 09:53:15 -0700

Hi Jana,

The great thing about your email is that is suggests a specific strategy
for how to choose next features. I'm especially excited about emphasizing
wire-visible stuff.

IMO this would emphasize two things: continuing to refine the public
header, and handling more paths in the handshake (i.e. HRR, stateless
reset, transport parameters, etc). In particular, if we can settle on these
items and move GQUIC onto it we should be pretty safe.

I'm not so sure about HTTP/2 mapping when there's so much about basic
stream semantics still up in the air. It has little to do with middlebox
ossification as well.

At the very least, I'll present this item for discussion in Prague. I'll
mull over changing the draft beforehand.

On Sat, Jul 8, 2017 at 9:45 AM, Jana Iyengar <[email-masked]> wrote:

>> Side note: Personally, I don’t particularly like the terminology of
>> first, second etc for the reason that it gets hard to rev/iterate on a
>> particular one. However, perhaps it was purposely chosen to disambiguate
>> from the I-D versions?
>>
>>
>>

---

From: Martin Duke <[email-masked]>
Date: Mon, 10 Jul 2017 09:55:26 -0700

I'm not sure how "performance analysis" is going to function in the absence
of loss recovery or congestion control. An alternate approach to
implementations is to tackle the big performance drivers first, presumably
loss recovery, congestion control, and streaming to prevent HOL blocking.
However, this would run directly opposite to Jana's suggestion to lock down
the wire image to prevent ossification.

---

From: Ian Swett <[email-masked]>
Date: Mon, 10 Jul 2017 20:06:29 -0400

Agreed, performance analysis is going to be useless in the absence of loss
recovery and congestion control.  Presumably anyone deploying this at scale
would implement the recovery draft in a relatively complete manner, but
that doesn't mean everyone has to do it.

But there's nothing interesting to measure with no application.

On Mon, Jul 10, 2017 at 12:55 PM, Martin Duke <[email-masked]>
wrote:

> I'm not sure how "performance analysis" is going to function in the
> absence of loss recovery or congestion control. An alternate approach to
>> >> > classifiers for gQUIC, and we need to move quickly and push
>> IETF-QUIC so
>> >> > we
>> >> > can test that IETF-QUIC is deployable. I fear that the longer we
>> take,
>> >> > the

---

From: Martin Duke <[email-masked]>
Date: Thu, 13 Jul 2017 16:46:05 -0700

Alright, I updated the second implementation draft significantly.

There are now two strategies: "Lock down the wire image" and "do what we
need to allow useful performance testing". I much prefer the former but it
is worth discussing, since people appear to be interested in both.

It's also clear (at least to me) that we need to do basic stream life-cycle
stuff in either case, so that has moved into the "must include" category.

Martin

On Mon, Jul 10, 2017 at 5:06 PM, Ian Swett <[email-masked]> wrote:

>>> >> > particularly easy for folks to deploy QUIC with that code. I think
>>> we
>>> >> > need
>>> >> > 1. work out the wire-visible invariants and finalize all of those
>>> for
>>> >> > the

---

From: Ian Swett <[email-masked]>
Date: Thu, 13 Jul 2017 20:28:11 -0400

Thanks for the update.  I would suggest a third potential option, which is
a mix of what you have with a small clarification(in bold):

   -

   Further revisions to mechanisms in the First Implementation Draft (e.g.
   changes to the public header format, connection close).
   -

   Transport Parameter Exchange. At the very least, the four parameters
   specified as MUST in the draft.
   -

   Address validation and HelloRetryRequest
   -

   An HTTP/2 application to require multiple streams *(with stateless HPACK
   compression, no QPACK, QCRAM, etc) and no server push*.

Any implementations that deploy at any scale must also do:

   -

   Loss Recovery beyond the exising 1-RTO retransmissions. (I believe this
   includes a number of concepts that are extensively tested in TCP and has
   low interoperability concerns).
   -

   Congestion Control

The reasoning being that both stateless reset and 0RTT are a fair bit of
work to get right based on my experience, and are not critical to having a
useful QUIC application.

On Thu, Jul 13, 2017 at 7:46 PM, Martin Duke <[email-masked]>
wrote:

> It's also clear (at least to me) that we need to do basic stream
> life-cycle stuff in either case, so that has moved into the "must include"
> category.
>
> Martin
>
>> Agreed, performance analysis is going to be useless in the absence of
>> loss recovery and congestion control.  Presumably anyone deploying this at
>> scale would implement the recovery draft in a relatively complete manner,
>> but that doesn't mean everyone has to do it.
>>
>>>> >> > to move with some urgency, even if we don't change everything
>>>> about QUIC
>>>> >> > to
>>>> >> > 2. allow impls to make serious progress towards a basic HTTP
>>>> mapping
>>>> >> > over

---

From: Martin Duke <[email-masked]>
Date: Thu, 13 Jul 2017 17:42:21 -0700

Stateless reset (NOT stateless Reject)  is in option 1 because it is a wire
image issue. 0-RTT is in option 2 because I think it's an important to
performance.

On Thu, Jul 13, 2017 at 5:28 PM, Ian Swett <[email-masked]> wrote:

>    Further revisions to mechanisms in the First Implementation Draft
>    (e.g. changes to the public header format, connection close).
>    -
>
>    An HTTP/2 application to require multiple streams *(with stateless
>    HPACK compression, no QPACK, QCRAM, etc) and no server push*.
>
>
>    Loss Recovery beyond the exising 1-RTO retransmissions. (I believe
>    this includes a number of concepts that are extensively tested in TCP and
>    has low interoperability concerns).
>    -
>
>    Congestion Control
>
>
>>>>> >> > particularly easy for folks to deploy QUIC with that code. I
>>>>> think we
>>>>> >> > need
>>>>> >> > QUIC. We can punt on header compression (QPACK/QCRAM), but
>>>>> perhaps test
>>>>> >> > a

---

From: Ian Swett <[email-masked]>
Date: Thu, 13 Jul 2017 20:46:37 -0400

Sorry, I am still not used to the new name.  I meant to say stateless
reject is a substantial amount of work.  Stateless reset should definitely
be in scope.

And I'm now starting to think one of them should be renamed.

On Thu, Jul 13, 2017 at 8:42 PM, Martin Duke <[email-masked]>
wrote:

> Stateless reset (NOT stateless Reject)  is in option 1 because it is a
> wire image issue. 0-RTT is in option 2 because I think it's an important to
> performance.
>
>> Thanks for the update.  I would suggest a third potential option, which
>> is a mix of what you have with a small clarification(in bold):
>>
>>
>>    -
>>
>>>>>> >> > 1. work out the wire-visible invariants and finalize all of
>>>>>> those for
>>>>>> >> > the

---

From: Martin Duke <[email-masked]>
Date: Thu, 13 Jul 2017 17:47:53 -0700

I'm a little concerned that a blend of the strategies leaves us with
something that still allows ossification and isn't quite enough for decent
performance testing. But I did add your further clarifications about
HTTP/2, and the "at scale" provision.

Is your option 3 the MVP for something you'd like to do with this iteration?

---

From: Ian Swett <[email-masked]>
Date: Thu, 13 Jul 2017 20:57:02 -0400

I think it is an MVP of HTTP over QUIC, and one I believe Google would be
willing to deploy at some scale.

That's not to say I'm opposed to the other two options, but I have a
preference for something I believe can run real applications with
reasonable performance, even if it's not ideal.

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 14 Jul 2017 10:33:36 +0900

2017-07-14 9:57 GMT+09:00 Ian Swett <[email-masked]>:
> preference for something I believe can run real applications with reasonable
> performance, even if it's not ideal.

+1.

I agree that think that sending multiple HTTP requests / responses
over QUIC, with stateless HPACK would be a nicely balanced approach
that can be used deployed at some scale as well as one that can be
used for preliminary performance measurements.

>> performance testing. But I did add your further clarifications about HTTP/2,
>> and the "at scale" provision.
>>
>>> An HTTP/2 application to require multiple streams (with stateless HPACK
>>> compression, no QPACK, QCRAM, etc) and no server push.
>>>
>>>
>>> includes a number of concepts that are extensively tested in TCP and has low
>>> interoperability concerns).
>>>
>>> Congestion Control
>>>
>>>

-- 
Kazuho Oku

---

From: Martin Duke <[email-masked]>
Date: Fri, 14 Jul 2017 09:50:58 -0700

OK, I understand now. I've added a third option as you described. But I
omitted

as something that might not be entirely relevant to getting something
deployed, in the interests of reducing scope. If that's a big mistake, let
me know.

On Thu, Jul 13, 2017 at 6:33 PM, Kazuho Oku <[email-masked]> wrote:

> >>>>>>> >> > I've been thinking about this, and I'm starting to think that
> we
> >>>>>>> >> > should
> >>>>>>> >> > pressures. The availability of the Chromium implementation
> makes
> >>>>>>> >> > it
> >>>>>>> >> > second impl draft. We know that there are some middleboxes
> that
> >>>>>>> >> > already
> >>>>>>> >> > have

---

From: Ian Swett <[email-masked]>
Date: Fri, 14 Jul 2017 13:01:24 -0400

Thanks Martin!

On Fri, Jul 14, 2017 at 12:50 PM, Martin Duke <[email-masked]>
wrote:

>> (e.g.
>> this
>> >>> On Thu, Jul 13, 2017 at 7:46 PM, Martin Duke <[email-masked]
>> >
>> >>> wrote:
>> >>>>
>> >>>>>>> > On Sun, Jul 9, 2017 at 5:39 PM, Kazuho Oku <[email-masked]
>> >
>> >>>>>>> > wrote:
>> >>>>>>> >>
>> >>>>>>> >> > I've been thinking about this, and I'm starting to think
>> that we
>> >>>>>>> >> > should
>> >>>>>>> >> > I'm hearing about increasing deployments of gQUIC, largely
>> due
>> >>>>>>> >> > to market
>> >>>>>>> > Interesting idea. One challenge with performance analysis is
>> that
>> >>>>>>> > it'll be a
>> >>>>>>> > bit of an apples to oranges comparison. QUIC will be doing
>> HTTP/1
>> >>>>>>> > (without

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 14 Jul 2017 10:45:11 -0700

I think it would be good to declare another "implementation draft" very
soon, with roughly the contents of -05. What I see people doing now is some
mix of -04 and -05(pre), and informally coordinating. It would probably be
easier if we just published -05 and said "do -05". I totally don't care if
we call it the second implementation draft or the first implementation
draft bis or anything else.

-Ekr

---

From: Martin Duke <[email-masked]>
Date: Fri, 14 Jul 2017 17:17:55 -0700

If there are some improvements in -05, that's fine with me, but that's a
bit orthogonal to the purpose of this document. The first interop is
attached to a specific set of features, which might involve more than one
draft.

This document is trying to figure out what the next set of features should
be. It should also drive which open issues we should prioritize.

On Fri, Jul 14, 2017 at 10:45 AM, Eric Rescorla <[email-masked]> wrote:

>>> >>>>>> On Mon, Jul 10, 2017 at 12:32 AM, Kazuho Oku <[email-masked]
>>> >
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> > On Sun, Jul 9, 2017 at 5:39 PM, Kazuho Oku <
>>> [email-masked]>
>>> >>>>>>> > wrote:
>>> >>>>>>> >>
>>> >>>>>>> >> > control to later. This forces us to try and finalize the
>>> HTTP
>>> >>>>>>> >> > mapping