Subject: Our Schedule

From: Mark Nottingham <[email-masked]>
Date: Sat, 24 May 2014 09:23:30 +1000

In a little less than two weeks, we have an interim in NYC.

We’ve designated that as an interop event for draft -12 (and associated specs), and so far we seem to have good representation (reminder: if you haven’t already, please update <https://github.com/http2/http2-spec/wiki/Implementations>).

Our agenda for the interim is to go (yet again) through our open issues:
  https://github.com/http2/http2-spec/issues?labels=design&page=1&state=open

Assuming we can close those at or soon after the interim, we’ll issue another draft around mid-June. 

My current thinking — again, assuming we leave the interim with no open issues (or a plan to close them) — is to:

* Mark that draft as an Implementation Draft, with the interop targeted at our meeting in Toronto
* Issue a Working Group Last Call with a deadline in mid-July
* Based upon WGLC feedback and interop feedback, make a decision about submitting the documents to the IESG at our Toronto meeting.

The IETF LC will last at least two weeks, and then it goes to the IESG for their consideration. Personally, I think that if we follow the plan above, we have every chance of having an RFC well before the end of the calendar year.

All of this is contingent on being able to close out our issues in a timely manner, and not discovering significant new issues in WGLC, of course. If that happens, we’re going to need more rounds of interop, we’ll likely need more meetings, and the schedule will slip (perhaps significantly).

We’ll discuss all of this in more detail in NYC, but I wanted to give people a heads-up on the list as to where I think we’re heading.

So far, I’ve heard strong preferences across the board for keeping the schedule tight, and the above reflects that.  Going into NYC and Toronto, we need to keep this in mind as we make decisions; every change we make and especially every feature we add has the potential to delay us.

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 24 May 2014 11:43:01 +0200

Mark et al,

Having only recently re-engaged with this process, the jetty team
represents both fresh eyes looking at the draft, but also a wealth of
experience in implementing HTTP, Websocket and SPDY.   I do not see a draft
that is anywhere near to being ready for LC.

At the very least in the WG there currently exists a level confusion on
fundamental matters that should not result from a clear specification.  But
my feel is that many current threads actually represent a lingering level
of dissatisfaction.    Perhaps these issues have already been put to
consensus calls, so the resistance has now be reduced to the occasional
"+1", but the fact that new eyes can read the draft and still raise such
concerns means that protocol is too complex; the draft insufficiently clear
or both!

Important issues for me include:

   - The state machine in section 5.1 is essentially a fantasy that
   describes an idealised protocol that the draft does not represent.   In my
   work I very frequently lookup the TCP/IP state machine diagram that is a
   great reference and rarely do I need to go beyond it to understand any
   issue I have. However, if developers with HTTP/2 problems try to use the
   state machine in 5.1 as a reference, they are only going to be more
   confused. For example, the ES transitions are not atomic and the closed
   state is described as "the terminal state", but is then followed by 5
   paragraphs of dense complex text describing exceptions and how some frames
   received in closed state have to be handled!  I have posted what I think is
   the real state machine (
   http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/att-0720/http2state.txt)
   and it is a much more complex beastie.

   - There is also ongoing objections raised to hpack, both in it's
   complexity and it's impact on the rest of the protocol.  Hpack has been
   designed to be streaming, but since common fields will only be emitted at
   the end, receivers are forced to buffer the entire header anyway.  There
   are significant concerns that hpack will be able to be correctly
   implemented and/or tested and there is a high probability of many incorrect
   implementations that may expose the protocol to security and/or DOS issues.

   - Users of the protocol, are able to send data as headers, which is
   unconstrained by size, flow control or segmentation.   Client-side and
   Server-side applications are bound to collude to exploit this to obtain an
   unfair share of any shared HTTP/2 connections/infrastructure, which will
   eventually break the fundamental paradigm around which the protocol is
   based and eventually force the use of multiple connections again.

   - There is no clear layering of the protocol. SPDY has a well defined
   framing layer that can be separately implemented and tested.  HTTP is only
   a semantic layered on top of that framing layer.  HTTP/2 should do the same
   and it is a MASSIVE mistake verging on hubris to give up this fundamental
   good protocol practise in the name of speculative efficiency gains.

I respect the IETF process and if all these issues have been raised before,
discussed and put to consensus, then I'm put myself on mute and get on with
implementing the protocol.   But the process does sometimes get it wrong
and we really don't want to have a HTTP/2.1 any time soon.  So perhaps for
a such an important protocol, rather than rush forward to LC and an RFC by
the end of the year, perhaps pause, step back for a while, solicit some
wider review and listen to some new voices and be prepared to re-evaluate
if all the complexity is really justified?

regards

On 24 May 2014 01:23, Mark Nottingham <[email-masked]> wrote:

> We’ve designated that as an interop event for draft -12 (and associated
> specs), and so far we seem to have good representation (reminder: if you
> haven’t already, please update <
> https://github.com/http2/http2-spec/wiki/Implementations>).
>
> My current thinking — again, assuming we leave the interim with no open
> issues (or a plan to close them) — is to:
>
> * Based upon WGLC feedback and interop feedback, make a decision about
> submitting the documents to the IESG at our Toronto meeting.
>
> The IETF LC will last at least two weeks, and then it goes to the IESG for
> their consideration. Personally, I think that if we follow the plan above,
> we have every chance of having an RFC well before the end of the calendar
> year.
>
> All of this is contingent on being able to close out our issues in a
> timely manner, and not discovering significant new issues in WGLC, of
> course. If that happens, we’re going to need more rounds of interop, we’ll
> likely need more meetings, and the schedule will slip (perhaps
> significantly).
>
> We’ll discuss all of this in more detail in NYC, but I wanted to give
> people a heads-up on the list as to where I think we’re heading.
>
> So far, I’ve heard strong preferences across the board for keeping the
> schedule tight, and the above reflects that.  Going into NYC and Toronto,
> we need to keep this in mind as we make decisions; every change we make and
> especially every feature we add has the potential to delay us.
>
> Regards,
>
>
> --

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: James M Snell <[email-masked]>
Date: Sat, 24 May 2014 06:51:41 -0700

+1. I will note that many of the issues have been raised before (e.g. HPACK
complexity) but few have been adequately dealt with, IMHO. I know there is
a strong desire to just finish, but I think rushing to last call in the
specs current state is folly.
On May 24, 2014 2:48 AM, "Greg Wilkins" <[email-masked]> wrote:

> I respect the IETF process and if all these issues have been raised
> before, discussed and put to consensus, then I'm put myself on mute and get
> on with implementing the protocol.   But the process does sometimes get it
> wrong and we really don't want to have a HTTP/2.1 any time soon.  So
> perhaps for a such an important protocol, rather than rush forward to LC
> and an RFC by the end of the year, perhaps pause, step back for a while,
> solicit some wider review and listen to some new voices and be prepared to
> re-evaluate if all the complexity is really justified?
>
>
> regards
>
>
>
>
>
>
>> The IETF LC will last at least two weeks, and then it goes to the IESG
>> for their consideration. Personally, I think that if we follow the plan
>> above, we have every chance of having an RFC well before the end of the
>> calendar year.
>>

---

From: Mike Sweet <[email-masked]>
Date: Sat, 24 May 2014 16:52:27 -0400

+1

Sent from my iPhone

> On May 24, 2014, at 9:51 AM, James M Snell <[email-masked]> wrote:
> 
> +1. I will note that many of the issues have been raised before (e.g. HPACK complexity) but few have been adequately dealt with, IMHO. I know there is a strong desire to just finish, but I think rushing to last call in the specs current state is folly.
> 
>> Having only recently re-engaged with this process, the jetty team represents both fresh eyes looking at the draft, but also a wealth of experience in implementing HTTP, Websocket and SPDY.   I do not see a draft that is anywhere near to being ready for LC.
>> 
>> At the very least in the WG there currently exists a level confusion on fundamental matters that should not result from a clear specification.  But my feel is that many current threads actually represent a lingering level of dissatisfaction.    Perhaps these issues have already been put to consensus calls, so the resistance has now be reduced to the occasional "+1", but the fact that new eyes can read the draft and still raise such concerns means that protocol is too complex; the draft insufficiently clear or both!
>> 
>> The state machine in section 5.1 is essentially a fantasy that describes an idealised protocol that the draft does not represent.   In my work I very frequently lookup the TCP/IP state machine diagram that is a great reference and rarely do I need to go beyond it to understand any issue I have. However, if developers with HTTP/2 problems try to use the state machine in 5.1 as a reference, they are only going to be more confused. For example, the ES transitions are not atomic and the closed state is described as "the terminal state", but is then followed by 5 paragraphs of dense complex text describing exceptions and how some frames received in closed state have to be handled!  I have posted what I think is the real state machine (http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/att-0720/http2state.txt) and it is a much more complex beastie.
>> 
>> There is also ongoing objections raised to hpack, both in it's complexity and it's impact on the rest of the protocol.  Hpack has been designed to be streaming, but since common fields will only be emitted at the end, receivers are forced to buffer the entire header anyway.  There are significant concerns that hpack will be able to be correctly implemented and/or tested and there is a high probability of many incorrect implementations that may expose the protocol to security and/or DOS issues.
>> 
>> Users of the protocol, are able to send data as headers, which is unconstrained by size, flow control or segmentation.   Client-side and Server-side applications are bound to collude to exploit this to obtain an unfair share of any shared HTTP/2 connections/infrastructure, which will eventually break the fundamental paradigm around which the protocol is based and eventually force the use of multiple connections again.
>> 
>> There is no clear layering of the protocol. SPDY has a well defined framing layer that can be separately implemented and tested.  HTTP is only a semantic layered on top of that framing layer.  HTTP/2 should do the same and it is a MASSIVE mistake verging on hubris to give up this fundamental good protocol practise in the name of speculative efficiency gains.
>> I respect the IETF process and if all these issues have been raised before, discussed and put to consensus, then I'm put myself on mute and get on with implementing the protocol.   But the process does sometimes get it wrong and we really don't want to have a HTTP/2.1 any time soon.  So perhaps for a such an important protocol, rather than rush forward to LC and an RFC by the end of the year, perhaps pause, step back for a while, solicit some wider review and listen to some new voices and be prepared to re-evaluate if all the complexity is really justified?
>> 
>> 
>> regards
>> 
>> 
>> 
>> 
>> 
>>

---

From: Roland Zink <[email-masked]>
Date: Sat, 24 May 2014 22:57:58 +0200

+1

On 24.05.2014 22:52, Mike Sweet wrote:
> +1
>
> Sent from my iPhone
>
> On May 24, 2014, at 9:51 AM, James M Snell <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>> +1. I will note that many of the issues have been raised before (e.g. 
>> HPACK complexity) but few have been adequately dealt with, IMHO. I 
>> know there is a strong desire to just finish, but I think rushing to 
>> last call in the specs current state is folly.
>>
>> On May 24, 2014 2:48 AM, "Greg Wilkins" <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>>
>>     Mark et al,
>>
>>     represents both fresh eyes looking at the draft, but also a
>>     wealth of experience in implementing HTTP, Websocket and SPDY.  
>>     I do not see a draft that is anywhere near to being ready for LC.
>>
>>     At the very least in the WG there currently exists a level
>>     confusion on fundamental matters that should not result from a
>>     clear specification.  But my feel is that many current threads
>>     actually represent a lingering level of dissatisfaction.   
>>     Perhaps these issues have already been put to consensus calls, so
>>     the resistance has now be reduced to the occasional "+1", but the
>>     fact that new eyes can read the draft and still raise such
>>     concerns means that protocol is too complex; the draft
>>     insufficiently clear or both!
>>
>>       * The state machine in section 5.1 is essentially a fantasy
>>         that describes an idealised protocol that the draft does not
>>         represent.   In my work I very frequently lookup the TCP/IP
>>         state machine diagram that is a great reference and rarely do
>>         I need to go beyond it to understand any issue I have.
>>         However, if developers with HTTP/2 problems try to use the
>>         state machine in 5.1 as a reference, they are only going to
>>         be more confused. For example, the ES transitions are not
>>         atomic and the closed state is described as "the terminal
>>         state", but is then followed by 5 paragraphs of dense complex
>>         text describing exceptions and how some frames received in
>>         closed state have to be handled!  I have posted what I think
>>         is the real state machine
>>         (http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/att-0720/http2state.txt)
>>       * There is also ongoing objections raised to hpack, both in
>>         it's complexity and it's impact on the rest of the protocol. 
>>         Hpack has been designed to be streaming, but since common
>>         fields will only be emitted at the end, receivers are forced
>>         to buffer the entire header anyway.  There are significant
>>         concerns that hpack will be able to be correctly implemented
>>         and/or tested and there is a high probability of many
>>         incorrect implementations that may expose the protocol to
>>         security and/or DOS issues.
>>
>>       * Users of the protocol, are able to send data as headers,
>>         which is unconstrained by size, flow control or
>>         segmentation.   Client-side and Server-side applications are
>>         bound to collude to exploit this to obtain an unfair share of
>>         any shared HTTP/2 connections/infrastructure, which will
>>         eventually break the fundamental paradigm around which the
>>         protocol is based and eventually force the use of multiple
>>         connections again.
>>
>>       * There is no clear layering of the protocol. SPDY has a well
>>         defined framing layer that can be separately implemented and
>>         tested.  HTTP is only a semantic layered on top of that
>>         framing layer. HTTP/2 should do the same and it is a MASSIVE
>>         mistake verging on hubris to give up this fundamental good
>>         protocol practise in the name of speculative efficiency gains.
>>
>>     I respect the IETF process and if all these issues have been
>>     raised before, discussed and put to consensus, then I'm put
>>     myself on mute and get on with implementing the protocol.   But
>>     the process does sometimes get it wrong and we really don't want
>>     to have a HTTP/2.1 any time soon.  So perhaps for a such an
>>     important protocol, rather than rush forward to LC and an RFC by
>>     the end of the year, perhaps pause, step back for a while,
>>     solicit some wider review and listen to some new voices and be
>>     prepared to re-evaluate if all the complexity is really justified?
>>
>>
>>     regards
>>
>>
>>
>>
>>
>>
>>     On 24 May 2014 01:23, Mark Nottingham <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>         We’ve designated that as an interop event for draft -12 (and
>>         associated specs), and so far we seem to have good
>>         representation (reminder: if you haven’t already, please
>>         update
>>         <https://github.com/http2/http2-spec/wiki/Implementations>).
>>
>>         Assuming we can close those at or soon after the interim,
>>         we’ll issue another draft around mid-June.
>>
>>         My current thinking — again, assuming we leave the interim
>>         with no open issues (or a plan to close them) — is to:
>>
>>         * Mark that draft as an Implementation Draft, with the
>>         interop targeted at our meeting in Toronto
>>         * Based upon WGLC feedback and interop feedback, make a
>>         decision about submitting the documents to the IESG at our
>>         Toronto meeting.
>>
>>         The IETF LC will last at least two weeks, and then it goes to
>>         the IESG for their consideration. Personally, I think that if
>>         we follow the plan above, we have every chance of having an
>>         RFC well before the end of the calendar year.
>>
>>         All of this is contingent on being able to close out our
>>         issues in a timely manner, and not discovering significant
>>         new issues in WGLC, of course. If that happens, we’re going
>>         to need more rounds of interop, we’ll likely need more
>>         meetings, and the schedule will slip (perhaps significantly).
>>
>>         We’ll discuss all of this in more detail in NYC, but I wanted
>>         to give people a heads-up on the list as to where I think
>>         we’re heading.
>>
>>         So far, I’ve heard strong preferences across the board for
>>         keeping the schedule tight, and the above reflects that.
>>          Going into NYC and Toronto, we need to keep this in mind as
>>         we make decisions; every change we make and especially every
>>         feature we add has the potential to delay us.
>>
>>         Regards,
>>
>>
>>         --
>>     Greg Wilkins <[email-masked] <mailto:[email-masked]>>

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 May 2014 15:10:38 +1000

Hi Greg,

On 24 May 2014, at 7:43 pm, Greg Wilkins <[email-masked]> wrote:

> 
> Mark et al,
> 
> Having only recently re-engaged with this process, the jetty team represents both fresh eyes looking at the draft, but also a wealth of experience in implementing HTTP, Websocket and SPDY.

Thanks for the feedback — fresh eyes are indeed very important and useful.

> At the very least in the WG there currently exists a level confusion on fundamental matters that should not result from a clear specification.  But my feel is that many current threads actually represent a lingering level of dissatisfaction.
> 
> Perhaps these issues have already been put to consensus calls, so the resistance has now be reduced to the occasional "+1", but the fact that new eyes can read the draft and still raise such concerns means that protocol is too complex; the draft insufficiently clear or both!

Both complexity and clarify of specification have been and remain a concern of mine, but so far we’ve had good interoperability between the many implementations on record. That includes not a small number of implementations from people whose native language is not English; I take that as an indication we’re doing pretty well.

Having said that, there’s always room for improvement. Making things better means either changing the protocol itself, or improving how it’s written down.

Changing the protocol is still possible, but we’ve heard again and again that people are becoming more resistant to changes as we go on, so getting consensus on a significant change is going to take a clear proposal and agreement that it improves things considerably.

On the other hand, improving the draft is relatively easy to do, provided we have good proposals for text.

We’re clearly not going to make everyone satisfied with this specification; the best we can do is make everyone more-or-less equally dissatisfied. Right now, I’m hearing dissatisfaction from you and others about spec complexity at the same time I’m hearing dissatisfaction from others about schedule slips...

> 	• The state machine in section 5.1 is essentially a fantasy that describes an idealised protocol that the draft does not represent.   In my work I very frequently lookup the TCP/IP state machine diagram that is a great reference and rarely do I need to go beyond it to understand any issue I have. However, if developers with HTTP/2 problems try to use the state machine in 5.1 as a reference, they are only going to be more confused. For example, the ES transitions are not atomic and the closed state is described as "the terminal state", but is then followed by 5 paragraphs of dense complex text describing exceptions and how some frames received in closed state have to be handled!  I have posted what I think is the real state machine (http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/att-0720/http2state.txt) and it is a much more complex beastie.

Thanks for making a concrete proposal here. I’ve created <https://github.com/http2/http2-spec/issues/484> to track this.

> 	• There is also ongoing objections raised to hpack, both in it's complexity and it's impact on the rest of the protocol.  Hpack has been designed to be streaming, but since common fields will only be emitted at the end, receivers are forced to buffer the entire header anyway.

I’d characterise the group’s current stance as acknowledging that HPACK is more complex than we’d like, in that there isn’t an off-the-shelf algorithm that we can use (as was the case with gzip). We’ve discussed this pretty extensively a number of times, and so far the decision has been to stay with it.

> There are significant concerns that hpack will be able to be correctly implemented and/or tested and there is a high probability of many incorrect implementations that may expose the protocol to security and/or DOS issues.

HPACK has been implemented interoperably by many. It’s also been reviewed for security issues, and so far the risk has been felt to be acceptable.

If you have a concrete issue regarding HPACK security or its operation, please bring it up; making sweeping, predictive statements doesn’t really move us forward.

> 	• Users of the protocol, are able to send data as headers, which is unconstrained by size, flow control or segmentation.   Client-side and Server-side applications are bound to collude to exploit this to obtain an unfair share of any shared HTTP/2 connections/infrastructure, which will eventually break the fundamental paradigm around which the protocol is based and eventually force the use of multiple connections again.

Can you explain this a bit more, perhaps with an example?

> 	• There is no clear layering of the protocol. SPDY has a well defined framing layer that can be separately implemented and tested.  HTTP is only a semantic layered on top of that framing layer.  HTTP/2 should do the same and it is a MASSIVE mistake verging on hubris to give up this fundamental good protocol practise in the name of speculative efficiency gains.

Greg, I don’t think slinging around phrases like “MASSIVE mistake verging on hubris” is helpful. Many have pointed out just the opposite — that trying to create a neatly layered architecture in this case is boiling an ocean, a la BEEP and WS-*.

Again, if you have a proposal for layering, we can consider that. 

> I respect the IETF process and if all these issues have been raised before, discussed and put to consensus, then I'm put myself on mute and get on with implementing the protocol.   But the process does sometimes get it wrong and we really don't want to have a HTTP/2.1 any time soon.
> 
> So perhaps for a such an important protocol, rather than rush forward to LC and an RFC by the end of the year, perhaps pause, step back for a while, solicit some wider review and listen to some new voices and be prepared to re-evaluate if all the complexity is really justified?

I’d go further and say that the process often gets it more wrong than right, and that predicting success of a standard is notoriously difficult. IME, holding a specification up in the process does not necessarily make it any better, and can often make it worse. 

The overwhelming preference expressed in the WG so far has been to work to a tight schedule. HTTP/3 has already been discussed a bit, because we acknowledge that we may not get everything right in HTTP/2, and there are some things we haven’t been able to do yet. As long as the negotiation mechanisms work out OK, that should be fine.

In other words, while there was a ~15 year gap between HTTP/1.1 and HTTP/2, it’s very likely that the next revision will come sooner. While we don’t want to needlessly rev the protocol, we also don’t want to turn this into a five+ year effort to design the perfect protocol — if we get it wrong, we can learn from those mistakes and correct them.

There are (at least) two months before we’re talking about handing these documents to the IESG. That’s time that you can spend helping us to clarify the specifications and raise concrete technical issues. As I said, the schedule listed in my mail was an ideal — it’s very possible we’ll come in later than that, especially if substantive issues come in.

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 06:34:25 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>The overwhelming preference expressed in the WG so far has been to work 
>to a tight schedule. HTTP/3 has already been discussed a bit, because we 
>acknowledge that we may not get everything right in HTTP/2, and there 
>are some things we haven't been able to do yet.

This is probably the most damning thing in this entire fiasco:

The Mythical Man-Month was published in 1975, and without even
the slightest hint of room for exceptions, it told us to:

	"Always throw the prototype away."

The WG took the prototype SPDY was, before even completing its
previous assignment, and wasted a lot of time and effort trying to
goldplate over the warts and mistakes in it.

And rather than "ohh, we get HTTP/2.0 almost for free", we found
out that there are numerous hard problems that SPDY doesn't even
get close to solving, and that we will need to make some simplifications
in the evolved HTTP concept if we ever want to solve them.

Now even the WG chair publically admits that the result is a qualified
fiasco and that we will have to replace it with something better
"sooner".

So what exactly do we gain by continuing ?

Wouldn't we get a better result from taking a much deeper look
at the current cryptographic and privacy situation, rather than
publish a protocol with a cryptographic band-aid which doesn't solve
the problems and gets in the way in many applications ?

Isn't publishing HTTP/2.0 as a "place-holder" is just a waste of
everybodys time, and a needless code churn, leading to increased
risk of security exposures and failure for no significant gains ?

Wouldn't it be faster to go straight after the real goal, an
protocol which CAN replace HTTP/1.1 in all scenarios and be
an improvement in ALL scenarios ?

Please admit defeat, and Do The Right Thing.

I move that we:

  * Credit SPDY as a very good and worthwhile prototype,

  * Acknowlege that SPDY overwhelmingly showed that there are a lot
    of room for improvement over HTTP/1.1,

  * But cognizant of the problems we've run into, and in respect
    for Fred Brooks sage advice we throw it out, now that we have
    learned all that we can from it.

and
 
  * Immediately start the design a successor protocol to
    HTTP/1.1, which through necessary simplifications of HTTP
    semantics and based on what we learned from the SPDY prototype,
    will become better than HTTP/1.1 for all uses and users.

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Yoav Nir <[email-masked]>
Date: Mon, 26 May 2014 10:15:19 +0300

> an improvement in ALL scenarios ?

Hi

I think that would be a fool’s errand. If SPDY has shown us anything, it is that HTTP/1.1 was not optimized for the web(*).  Neither SPDY nor HTTP/2 provide much advantage to the numerous other uses that have cropped up for HTTP: file transfer, web services, streaming media.

In the old days we had different protocols for different use cases. We had FTP and SSH and various protocols for RPC. Placing all our networking needs over HTTP was driven by the ubiquitous availability of HTTP stacks, and the need to circumvent firewalls. I don’t believe a single protocol can be optimal in all scenarios. So I believe we should work on the one where the pain is most obvious - the web - and avoid trying to solve everybody else’s problem. 

If HTTP/1.1 is not optimal for downloading 4 GB OS updates, let them create their own successor to HTTP/1.1, or use FTP, or use HTTP/2 even though it’s not optimal, just like they do today. You can’t optimize for everything at once.

Yoav

(*) I’m using “web” here in the narrow sense that involves a user and a browser, not an application that uses a web service to guess my location and get a weather report in JSON so it can display a picture of some flowers in the rain as a background for my phone.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 07:27:26 +0000

>I think that would be a fool's errand.
>[...]
>I don't believe a single protocol can be optimal in all scenarios.

I wrote "improvement", not "optimal".

---

From: David Krauss <[email-masked]>
Date: Mon, 26 May 2014 15:50:34 +0800

> If HTTP/1.1 is not optimal for downloading 4 GB OS updates, let them create their own successor to HTTP/1.1, or use FTP, or use HTTP/2 even though it’s not optimal, just like they do today. You can…

Then those protocols will just be firewalled too. The user fundamentally needs an encrypted connection that doesn’t reveal what kind of data is being transmitted. Although, I suppose, any protocol or server that supports upgrade of an HTTP/1.1 connection will do.

Also, back in those days we had client applications for each protocol. Today we do everything through the browser, not only because it solves end-user portability issues, but because specialized client software is too vulnerable. If browsers end up with internal fragmentation due to multiple upgrade protocols, then security becomes that much harder. 

Just to point out, the Web makes due without fancy headers. A few kilobytes at the start of a stream will do, although compression is certainly nice.

Segmentation likewise might as well be a “flush” directive, as for improving web apps. Although the HTTP/2 semantics already seem ideal for this application, it’s not clear whether it is abusive for an application to use it where a flush is not specifically desired, nor whether flushing ever occurs.

---

From: Mike Belshe <[email-masked]>
Date: Mon, 26 May 2014 01:00:07 -0700

-1

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 May 2014 19:14:30 +1000

Poul-Henning,

On 26 May 2014, at 4:34 pm, Poul-Henning Kamp <[email-masked]> wrote:

> "sooner”.

This is a gross mischaracterisation of what I said. I appreciate that you have a flair for the dramatic, but kindly refrain from putting words in my mouth.

> Please admit defeat, and Do The Right Thing.

This isn’t a parliamentary body, nor are Roberts’ Rules in effect. We don’t need motions, we need good technical arguments.

>  * Credit SPDY as a very good and worthwhile prototype,

Done.

>    of room for improvement over HTTP/1.1,

I think we’re there.

>    learned all that we can from it.

I don’t hear the engineers who have put a year and a half of work into this effort talking about "defeat" nor about it being a “fiasco.” They’re a very good representation of HTTP implementers, and from what I’ve heard to date, they think we’re getting close to shipping. 

If you want to come and make technical proposals in good faith, ask questions, argue for specific approaches, etc. you are — as always — welcome to participate. 

>    will become better than HTTP/1.1 for all uses and users.

Changing the semantics of HTTP are out of scope for good reason; if we change semantics, it creates incompatibility with the existing Web. As much as many people would like to get rid of Cookies — something you’ve proposed many times — doing it in this effort would be counter-productive.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 May 2014 11:19:41 +0200

On 2014-05-26 09:15, Yoav Nir wrote:
> ...

Out of curiosity: exactly how is HTTP/1.1 (or 2) worse than FTP for 
large file transfers?

Best regards, Julian

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 26 May 2014 12:19:44 +0300

> I respect the IETF process and if all these issues have been raised before, discussed and put to consensus, then I'm put myself on mute and get on with implementing the protocol.   But the process…

Hi Greg

I don't think there is a rush here! 

On the contrary
due the fact that the protocol has been largely discussed in the mailing list and the f2f meetings and interims,
implemented and that people behind the implementations have had several chances to interop
issuing a Working Group Last Call with a deadline in mid-July is actually a reasonable thing to do

A WG LC period but sometime also only the awareness that the WG LC is approaching
is the best way to solicit comments from a larger audience:
people with a fresh eyes as well as people that have only recently re-engaged with this protocol design.
It is the right way to make sure that "no important concerns have been missed or misunderstood".

We have also be aware of the fact that there can be more then one WG LC.
I think that what Mark was describing in his mail, is only how the process will evolve if everything goes smoothly

br
Salvatore

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 09:46:09 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>I don't hear the engineers who have put a year and a half of work into 
>this effort talking about "defeat" nor about it being a "fiasco."
>They're a very good representation of HTTP implementers, and from what 
>I've heard to date, they think we're getting close to shipping. 

Even asking the wrong question to the wrong people, you will get
to the end of the useless answer eventually.

>If you want to come and make technical proposals in good faith, ask 
>questions, argue for specific approaches, etc. you are -- as always --
>welcome to participate. 

I stated my case, and was told that it was far more important to
goldplate SPDY on a rushed and unrealistic schedule, to gain a
little bit of speed, than to try to solve the problems with
scalability and privacy I pointed out.

I'll be happy to participate in worthwhile projects, but HTTP/2.0
has never been that for me.

>Changing the semantics of HTTP are out of scope for good reason; 

I think of those as really bad reasons.

>As much as many people would like to get rid of Cookies -- something 
>you've proposed many times -- doing it in this effort would be 
>counter-productive.

Counter-productive for *who* Mark ?

Counter-productive for FaceBook, Google, Microsoft, NSA and the
other mastodons who use cookies and other mistakes in HTTP
(ie: user-agent) to deconstruct our personal identities, across
the entire web ?

Even with "SSL/TLS everywhere", all those small blue 'f' icons
will still tell FaceBook all about what websites you have visited.

The "don't track" fiasco has shown conclusively, that there is
never going to be a good-faith attempt by these mastodons to
improve personal privacy:  It's against their business model.

And because this WG is 100% beholden to the privacy abusers and
gives not a single shit for the privacy abused, fixing the problems
would be "counter-productive".

If we cared about human rights, and privacy, being "counter-productive"
for the privacy-abusing mastodons would be one of our primary goals.

Poul-Henning

---

From: Yoav Nir <[email-masked]>
Date: Mon, 26 May 2014 12:48:24 +0300

On May 26, 2014, at 12:19 PM, Julian Reschke <[email-masked]> wrote:

> Out of curiosity: exactly how is HTTP/1.1 (or 2) worse than FTP for large file transfers?

Translation of text encoding (ASCII/EBCDIC, CR/LF/CRLF)
Listing directories
Getting or putting multiple files based on wildcard with or without user prompt
Slightly less verbose (very slightly…)
With old-style FTP, each resource got its own TCP connection, and so its own flow control. That one we are getting with HTTP/2.

We used to move files using FTP. We moved to HTTP not because it was better, and not because of a lack of client support (all OSes have an FTP client, and all browsers support ftp:// URIs), but because everybody installed a firewall and half of those blocked FTP, but they all let HTTP through.

Yoav

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 09:49:37 +0000

In message <[email-masked]>, Yoav Nir writes:

>We moved to HTTP not because it was 
>better, and not because of a lack of client support (all OSes have an 
>FTP client, and all browsers support ftp:// URIs), but because everybody 
>installed a firewall and half of those blocked FTP, but they all let 
>HTTP through.

That really was a design-mistake in FTP to begin with, you can't blame
that on HTTP.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 10:01:25 +0000

In message <[email-masked]>, Yoav Nir writes:

>What design mistake? 

Not having the DATA connection go in the same direction as the
command channel by default.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 May 2014 12:01:44 +0200

> Translation of text encoding (ASCII/EBCDIC, CR/LF/CRLF)

I seriously believe that's not needed anymore.

> Listing directories

RFC 4918.

> Getting or putting multiple files based on wildcard with or without user prompt

RFC 4918, plus a client that uses it properly.

> Slightly less verbose (very slightly…)

How is this relevant for *large* files?

> With old-style FTP, each resource got its own TCP connection, and so its own flow control. That one we are getting with HTTP/2.

You can do that in HTTP/1.1 as well, if you want.

> We used to move files using FTP. We moved to HTTP not because it was better, and not because of a lack of client support (all OSes have an FTP client, and all browsers support ftp:// URIs), but bec…

I hear that browsers makers consider to remove "ftp:" support.

The last time I used FTP was a few years ago, when I had to fix a bug in 
essentially unmaintained Mozilla code, that I had previously broken by 
improving the compliance of their URI parsing code.

Best regards, Julian

---

From: Yoav Nir <[email-masked]>
Date: Mon, 26 May 2014 13:00:11 +0300

What design mistake? 

Yoav

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 May 2014 20:21:59 +1000

> to the end of the useless answer eventually.

We’re supposed to ignore the implementers of Mozilla, Chrome, IE, IIS, Apache Traffic Server, Curl and a bunch more because you are the “right” person?

Again, technical arguments, please.

> has never been that for me.

That’s regrettable.

> for the privacy-abusing mastodons would be one of our primary goals.

I and many of the people in the WG care deeply about privacy, but introducing a version of HTTP that doesn’t support cookies means that that version of HTTP will be largely ignored. You might as well the Transport Area to require support for RFC2514.

If you have a realistic proposal that improves privacy and provides for a transition away from Cookies on the Web, I and many others would very much like to see it.

---

From: Yoav Nir <[email-masked]>
Date: Mon, 26 May 2014 13:27:55 +0300

WebDav is not the base HTTP. It’s an extension, and it might as well have been an application running over HTTP.

I agree that support for ftp is waning because the use is waning (although the RFC editor only lets you download the whole RFC repository over either FTP or rsync. Even the index is only available via FTP).

Anyway, my point is not that FTP is that great. It’s that if the claims that pages load twice as fast in HTTP/2 are true, we should not hold off until we see a benefit to every other use of HTTP.

Yoav

On May 26, 2014, at 1:01 PM, Julian Reschke <[email-masked]> wrote:

> The last time I used FTP was a few years ago, when I had to fix a bug in essentially unmaintained Mozilla code, that I had previously broken by improving the compliance of their URI parsing code.
> 
> Best regards, Julian

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 26 May 2014 22:38:51 +1200

On 26/05/2014 7:50 p.m., David Krauss wrote:
> 
> On 2014–05–26, at 3:15 PM, Yoav Nir wrote:
> 
>> In the old days we had different protocols for different use cases.
>> We had FTP and SSH and various protocols for RPC. Placing all our
>> networking needs over HTTP was driven by the ubiquitous
>> availability of HTTP stacks, and the need to circumvent firewalls.
>> I don’t believe a single protocol can be optimal in all scenarios.
>> So I believe we should work on the one where the pain is most
>> obvious - the web - and avoid trying to solve everybody else’s
>> problem.
>> 
>> If HTTP/1.1 is not optimal for downloading 4 GB OS updates, let
>> them create their own successor to HTTP/1.1, or use FTP, or use
>> HTTP/2 even though it’s not optimal, just like they do today. You
>> can’t optimize for everything at once.
> 
> Then those protocols will just be firewalled too. The user
> fundamentally needs an encrypted connection that doesn’t reveal what
> kind of data is being transmitted. Although, I suppose, any protocol
> or server that supports upgrade of an HTTP/1.1 connection will do.

The system over which that traffic travels needs a protocol that it can
trust. Trust of the user or trust of what data the protocol contains -
either one will do, both together would be better.

HTTP has held onto that trust in a small way by being open and routinely
inspected. Is now on the fast track to also losing that trust completely.

> 
> Also, back in those days we had client applications for each
> protocol. Today we do everything through the browser, not only
> because it solves end-user portability issues, but because
> specialized client software is too vulnerable. If browsers end up
> with internal fragmentation due to multiple upgrade protocols, then
> security becomes that much harder.
> 

A decade or two ago firewalls were being rolled out everywhere ASAP to
filter and take control of the packets and protocols being used in those
unprotected UDP/TCP ports 0 through 65535.

Today a different type of MITM are being rolled out ASAP to filter and
take back control of the protocols being run via port 80 and 443 on top
of HTTP and TLS.

Consider carefully how it got to the point where firewalls have such a
"bad" reputation that port based protocols are now designed as if the
firewall were an enemy? What does that say about how HTTP will be seen
and operating tomorrow?

We can already have signs; in the X-Powered-By header created explicitly
to bypass *web server* control mechanisms, in popularity of MITM
interception proxies, in popularity of SSL decryption proxies, ...

Working against security systems in a technical arms race only harms us all.

If the protocol has real benefits and shows that it can be trusted the
firewalls etc can be opened where relevant. Design a protocol that can
be trusted, not one that implicitly shows malicious intent by explicitly
avoiding peoples choice of security.

> Just to point out, the Web makes due without fancy headers. A few
> kilobytes at the start of a stream will do, although compression is
> certainly nice.
> 

Nothing has changed there. The overay application protocols which do not
use headers are better off with HTTP/2 even as they do not need much
complexity of HEADERS and none of CONTINUATION.

> Segmentation likewise might as well be a “flush” directive, as for
> improving web apps. Although the HTTP/2 semantics already seem ideal
> for this application, it’s not clear whether it is abusive for an
> application to use it where a flush is not specifically desired, nor
> whether flushing ever occurs.

Amos

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 10:50:49 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

We're wandering out into the weeds here Mark, lets get back to my
proposal:

HTTP/2.0 is not enough of an improvement to warrant going on the
standards track, and that it should be dropped for that reason,
rather than make a lot of people waste a lot of time and expose the
net to more new complex software with new bugs.

If and when we are going to attempt again, we should start out
trying to find out what problems should be addressed and which
shouldn't, but that is an entirely different disussion, and I'll
be happy to take that discussion at that time, with or without
HTTP/2.0 in the picture.

But right now, my argument is that HTTP/2.0 is not good enough
that we should push it on the net at large.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 May 2014 20:53:15 +1000

> that we should push it on the net at large.

Noted.

I suspect we’ll be discussing this more at our upcoming meeting, as well as on-list.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 May 2014 13:09:11 +0200

> WebDav is not the base HTTP. It’s an extension, and it might as well have been an application running over HTTP.

WebDAV is an optional part of HTTP, the same way things like 
authentication or range requests are.

> I agree that support for ftp is waning because the use is waning (although the RFC editor only lets you download the whole RFC repository over either FTP or rsync. Even the index is only available…

<http://www.rfc-editor.org/in-notes/rfc-index.txt>

<http://www.rfc-editor.org/in-notes/rfc-index.xml>

> Anyway, my point is not that FTP is that great. It’s that if the claims that pages load twice as fast in HTTP/2 are true, we should not hold off until we see a benefit to every other use of HTTP.

:-)

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 11:11:36 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>I suspect we'll be discussing this more at our upcoming meeting, as
>well as on-list.

That's all I ask for.

Complexity and sheer size carries a big penalty in software quality,
and we should not expend code lines and complexity unless the gains
will be worth it.

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 26 May 2014 13:35:22 +0200

Mark,

I accept that some of my comments and phrasing are not exactly
constructive.  But they are intended to be helpful.    My impression is
that the WG has been dealing with things, issue by issue and that in
isolation each decision makes sense, but when taken together the sequence
of individual issue resolutions have lead the protocol architecture away
for some good norms and new issues appear to keep coming.

My post was not intended to expand on any particular point of concern, but
rather to point out that I see many significant concerns all over the
protocol, that I believe have a common root in some architectural problems
such as the decision to break layering in order to support hpack.
I need to evaluate how I can best contribute to the process and I'm not
convinced that engaging in an issue by issue detailed debate will be
constructive if there is significant architectural review required.   The
risk is that you become too busy fighting crocodiles and forget that we
came here to drain the swamp!

Hence I took the opportunity of this schedule thread to question the
resolve of the WG to push through to LC and RFC on the basis of the current
draft.    If it is the case that an RFC based largely on draft 12 is
possible by the end of the year, then my best start fighting crocodiles
(and more keep coming out of the swamp).   But if there still is in some
chance of significant architectural change, then my best contribution may
be to point out that the presence of a large numbers of wildlife may
indicate some systematic problems - which is what I'm attempting to do.

So that it the main point I wish to make in this response... but I will
address some individual crocodiles point by point below (so reading on is
probably neither constructive nor helpful in the context of this thread).

On 26 May 2014 07:10, Mark Nottingham <[email-masked]> wrote:

> Hi Greg,
>
> On 24 May 2014, at 7:43 pm, Greg Wilkins <[email-masked]> wrote:Thanks
> for making a concrete proposal here. I’ve created <
> https://github.com/http2/http2-spec/issues/484> to track this.
>

I'll try to expand on the state diagram in the next few days with a more
compact version and some supporting text (however I think that it is a good
diagram in as much as it does not require very much supportive text -
closed actually means closed!).

>
> HPACK has been implemented interoperably by many. It’s also been reviewed
> for security issues, and so far the risk has been felt to be acceptable.
>
>
Another of my concerns with HPACK is that the encoder has many strategic
choices to make that will affect both efficiency and security.
Interoperability testing is not sufficient to test if correct strategic
decisions have been made nor if the implementations are robust under a wide
range of encoding strategies.

If you have a concrete issue regarding HPACK security or its operation,
> please bring it up; making sweeping, predictive statements doesn’t really
> move us forward.
>

See above for my justification for sweeping statements that are intended to
be helpful.  But other concerns I have for HPACK include that a single per
connection reference set is both an issue for parallel slowdown, but also
insufficient to well encode connections that aggregate streams from
different sources.

I do see that HPACK can be very efficient, but I have not yet found the
justification for why HTTP/2 needs to have such aggressive compression.
It is very easy to make some significant gains over HTTP/1.1, so going for
a less invasive compression would seam a more prudent step.   My experience
from SPDY is that we are going to get most of the gains from multiplexing,
reduced round trips and from push.   Such an aggressive compression
algorithm does not seam worth the risk of reducing the take up of those
other good attributes (dang - making sweeping predictive statements again).

>
> >       • Users of the protocol, are able to send data as headers, which
> is unconstrained by size, flow control or segmentation.   Client-side and
A core aspect of HTTP/2 is about fairly sharing a single HTTP connection.
In the initial usage, this is envisioned to be shared by streams from the
one web application, but there will also be wider sharing: between tabs on
a browser talking to the same service; between streams from different
clients aggregated onto a connection by an intermediary.   But data sent as
headers rather than in data frames is not subject to the fair share
mechanisms.  It is not flow controlled, nor can it be segmented and
interleaved with data from other streams.

Thus if there is contention for shared connections, then a huge incentive
exists for to utilise headers for data transfer rather than data
frames.      Initially application access to this is somewhat limited by
the browsers and to some extend by server header limits.    But we have
already seen that browsers are prepared to abuse the RFC to gain
performance advantages over competitors (see the connection limit arms
race) and browser/server venders are not the only gatekeepers for access to
shared infrastructure. If the incentive exists, then it is possible and
likely developers will create clients and servers that will collude to send
their applications data via headers rather than by data frames, so that
they may gain an unfair share of any common connections.

Simply put, one user may grab exclusive use of the entire HTTP/2 connection
capacity by sending a HEADERS frame an then continuing to stream data in
headers encoded as CONTINUATION frames forever.

> Again, if you have a proposal for layering, we can consider that.

I'm thinking that is probably the best way forward for me.  I'll see if I
can prepare a version of the draft that removes the worst of the layering
breaches and see if there is any support for resetting to that.  Perhaps
that effort will convince me that we are on the right path?

Sorry for being non-constructive and provoking a wider meta-discussion.
But I am trying to be helpful in the least disruptive way possible and
think that at the point of first call for the last call is a good time to
take a moment to re-evaluate the big picture.

cheers

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 May 2014 21:49:46 +1000

On 26 May 2014, at 9:35 pm, Greg Wilkins <[email-masked]> wrote:

> Sorry for being non-constructive and provoking a wider meta-discussion.   But I am trying to be helpful in the least disruptive way possible and think that at the point of first call for the last call is a good time to take a moment to re-evaluate the big picture.

Not at all a problem; like I said, the perspective is very welcome. I’m just trying to find ways in which we can productively incorporate your input.

Please keep pushing.

Regards,

---

From: Michael Sweet <[email-masked]>
Date: Mon, 26 May 2014 08:23:43 -0400

Greg,

I agree with your comments about HPACK, and I have made similar comments over the last year.  Everyone seems to be focusing on compression as a defining, mandatory feature of HTTP/2, but as you say multiplexing and push get you most of the way there.  If we leave out HPACK for 2.0 (or adopt a simpler HPACK) we can get some operational experience to determine whether further complexity is required or justified.

On May 26, 2014, at 7:35 AM, Greg Wilkins <[email-masked]> wrote:
> ...
> I do see that HPACK can be very efficient, but I have not yet found the justification for why HTTP/2 needs to have such aggressive compression.   It is very easy to make some significant gains over HTTP/1.1, so going for a less invasive compression would seam a more prudent step.   My experience from SPDY is that we are going to get most of the gains from multiplexing, reduced round trips and from push.   Such an aggressive compression algorithm does not seam worth the risk of reducing the take up of those other good attributes (dang - making sweeping predictive statements again).

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: "Richard Wheeldon (rwheeldo)" <[email-masked]>
Date: Mon, 26 May 2014 13:00:29 +0000

I’m curious as to what data you’re basing this on? I’m not saying you’re wrong – just looking for evidence. In particular data or results that I could use to sanity test our own implementations,

Richard

From: Greg Wilkins [mailto:[email-masked]]

My experience from SPDY is that we are going to get most of the gains from multiplexing, reduced round trips and from push

---

From: Cory Benfield <[email-masked]>
Date: Mon, 26 May 2014 14:09:45 +0100

On 26 May 2014 at 13:27:45, Michael Sweet ([email-masked]) wrote:
> ...
> If we leave out HPACK for 2.0 (or adopt a simpler HPACK) we can get some 
> operational experience to determine whether further complexity is required or 
> justified.

+1. I’d like to discuss the idea of simplifying or abandoning HPACK.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Mon, 26 May 2014 23:07:57 +0900

> interleaved with data from other streams.
​Flow control is not a mechanism to enforce fair share of the connection.
​Read http://tools.ietf.org/html/draft-ietf-httpbis-http2-12#section-5.2.2​

> Simply put, one user may grab exclusive use of the entire HTTP/2
> connection capacity by sending a HEADERS frame an then continuing to stream
> data in headers encoded as CONTINUATION frames forever.
>
>
​Server and intermediaries can respond 431 if incoming headers are too
large and refuse to forward to the shared backend connection. Receiving
CONTINUATION forever is unrealistic situation.

Best regards,
Tatsuhiro Tsujikawa​

---

From: James M Snell <[email-masked]>
Date: Mon, 26 May 2014 07:07:59 -0700

Mark has asked for technical arguments. My point of view is this:

>From day one, there has never been a strong technical argument *in favor*
of HPACK... At least not one that justifies the significant increase in
complexity. Header compression qualifies as a "nice to have" but is
certainly not critical to preserving http semantics or even the operation
of the framing protocol. HPACK should not be a normative requirement in
http/2. If our concern is about saving bytes on the wire, there are less
complicated ways to do so.

HPACK is too damned complex. Sure, there are very smart people all over the
world who will get it right but as we see with things like heartbleed, it
only takes one bug in one popular implementation to cause major problems
for everyone. If the complexity is not justified, the risk that comes along
with that complexity is even less so.

So far, in this thread, there have been quite a few voices willing to speak
up and question whether HPACK is the right choice. And while some of those
voices may have a "flair for the dramatic", I trust that they are very
intelligent voices that are worth listening too.

As far as I'm concerned, I'm strongly -1 on moving forward on the proposed
schedule with an http/2 that normatively requires HPACK.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 26 May 2014 14:24:56 +0000

>> Anyway, my point is not that FTP is that great. Its that if the
>>claims that pages load twice as fast in HTTP/2 are true, we should
>>not hold off until we see a benefit to every other use of HTTP.

Any benefit gained must be judged relative to the incremental
complexity and vulnerability to compromise or failure it carries.

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 26 May 2014 16:44:06 +0200

On 26 May 2014 15:00, Richard Wheeldon (rwheeldo) <[email-masked]>wrote:

>  I’m curious as to what data you’re basing this on? I’m not saying you’re
> wrong – just looking for evidence. In particular data or results that I
> could use to sanity test our own implementations,
>

Nothing very objective I'm afraid.

You can see the kind of results we see with SPDY in our 2012 javaone
presentation:  https://www.youtube.com/watch?v=4Ai_rrhM8gA

Don't  get me wrong, we certainly have application scenarios where our
throughput bottle neck is the size of HTTP headers.  Reducing HTTP header
size should be a priority of HTTP/2.  But I don't think we need to
sacrifice good protocol basics to achieve a world record in compression
efficiency.

cheers

---

From: Patrick McManus <[email-masked]>
Date: Mon, 26 May 2014 10:45:33 -0400

> complicated ways to do so.

I disagree. the fundamental value of http/2 lies in mux and priority, and
to enable both of those you need to be able to achieve a high level of
parallelism. Due to CWND complications the only way to do that on the
request path has been shown to be with a compression scheme. gzip
accomplished that but had a security problem - thus hpack. Other schemes
are plausible, and ones such as james's were considered, but some mechanism
is required.

this is well worn ground. Forgive me if I don't weigh in on every episode
re-run while we do the implementation work to actually get this tested and
deployed. Looking forward to testing with everyone some more.

-P

---

From: James M Snell <[email-masked]>
Date: Mon, 26 May 2014 08:17:45 -0700

If http/2 requires a header compression mechanism as complicated as HPACK,
then I'll stick with http/1.1. I'm willing to grant that *some* compression
mechanism will make things better, but I cannot accept that HPACK is the
best option... And it would seem that I'm not alone in that. Regardless,
the WG will push forward with whatever the WG chooses to push forward with,
I was merely offering my response to Mark's inquiry about the schedule.

---

From: Michael Sweet <[email-masked]>
Date: Mon, 26 May 2014 12:08:03 -0400

Richard,

I think the original SPDY white paper has some of this information, although I wasn't able to find all of the numbers broken out:

	http://dev.chromium.org/spdy/spdy-whitepaper

Table 3 pretty clearly shows the effect of latency on page load time. (Table 1 also, although you have to make assumptions about DSL being higher latency than Cable)  TCP connections basically account for 1.5 RTTs (SYN, SYN-ACK, ACK) and TLS handshakes are typically 2 RTTs.  For some really quick, basic benchmarking, here is what I see via my 80Mbps cable Internet service (which I regularly get the advertised bandwidth but generally have higher RTTs thanks to my geographic location) for loading the primary content of each of the named sites via HTTPS:

  Site                  Total   RTT     Conn    Resp Headers  Response HTML  gzip HTML (est)
  --------------------  ------  ------  ------  ------------  -------------  ------------
  www.apple.com          436ms    79ms   276ms     227 / 3ms  13066 / 157ms   3200 / 38ms
  www.cups.org           663ms   120ms   420ms     205 / 7ms   7803 / 236ms   2979 / 90ms
  www.google.ca          425ms    29ms   101ms    792 / 20ms  12295 / 304ms  4808 / 118ms
  www.msweet.org         711ms   104ms   364ms    217 / 13ms   5836 / 334ms  2119 / 121ms
  www.pwg.org            482ms    96ms   336ms     272 / 3ms  14606 / 143ms   4001 / 39ms

And then a separate run for HTTP:

  Site                  Total   RTT     Conn    Resp Headers  Response HTML  gzip HTML (est)
  --------------------  ------  ------  ------  ------------  -------------  ------------
  www.apple.com          224ms    81ms   121ms     227 / 2ms  13151 / 101ms   3206 / 24ms
  www.cups.org           441ms   132ms   198ms     205 / 7ms   7803 / 236ms   2979 / 90ms
  www.google.ca           89ms    26ms    39ms     791 / 4ms   12210 / 46ms   4790 / 18ms
  www.msweet.org         223ms    88ms   132ms     217 / 4ms    5836 / 87ms   2119 / 31ms
  www.pwg.org            212ms    93ms   139ms     272 / 2ms   14606 / 71ms   4001 / 19ms

For all of these sites I measured the average ping, connect/TLS handshake, and GET times, and then looked at the returned headers and data.  I did not specify "Accept-Encoding: gzip" in the requests, so the sizes are uncompressed HTML.  The "gzip" column shows the compression that gzip would provide on the HTML (and estimated transfer time).  I did not measure the time to transmit the HTTP GET request, which did not include any of the usual cookie headers (which for each of these sites appears to be up to about 1k worth of cookies).

While I understand I am not measuring the whole page load time (including dependent resources, which are often many times the size of the initial HTML document), these numbers *do* show the per-connection overhead and provide an easy comparison to response header and page content times and sizes.  Headers certainly appear to be in the noise, with significant improvement being possible simply by multiplexing multiple requests over a single connection and compressing the message body.

Now, I can understand the temptation to want to squeeze every last percent of bandwidth reduction possible and have the optimum protocol for web browsing, etc., but ordinary users are probably not going to notice the few milliseconds that get shaved off by compressing headers.  They definitely *will* notice whole second improvements in page load times, which is what multiplexing and message body compression will yield.

---

From: Michael Sweet <[email-masked]>
Date: Mon, 26 May 2014 12:35:14 -0400

Patrick,

On May 26, 2014, at 10:45 AM, Patrick McManus <[email-masked]> wrote:
> ...
> I disagree. the fundamental value of http/2 lies in mux and priority, and to enable both of those you need to be able to achieve a high level of parallelism. Due to CWND complications the only way to do that on the request path has been shown to be with a compression scheme. gzip accomplished that but had a security problem - thus hpack. Other schemes are plausible, and ones such as james's were considered, but some mechanism is required.

I see several key problems with the current HPACK:

1. The compression state is hard to manage, particularly for proxies.
2. HEADER frames hold up the show (issue #481)
3. There is no way to negotiate a connection without Huffman compression of headers (issue #485).

*If* we can come up with a header compression scheme that does not suffer from these problems, it might be worth the added complexity in order to avoid TCP congestion window issues.  But given that we are already facing 3.5 RTTs worth of latency just to negotiate a TLS connection I'm not convinced that compressing the request headers will yield a user-visible improvement in the speed of their web browsing experience.

---

From: Simone Bordet <[email-masked]>
Date: Mon, 26 May 2014 19:01:42 +0200

Hi,

I have discussed HTTP 2.0 status with Greg, and below you can find my thoughts.

On Sat, May 24, 2014 at 11:43 AM, Greg Wilkins <[email-masked]> wrote:
> There is no clear layering of the protocol. SPDY has a well defined framing
> layer that can be separately implemented and tested.  HTTP is only a
> semantic layered on top of that framing layer.  HTTP/2 should do the same

This is IMHO the primary mistake.
The framing should be very simple, define stream start and end, and be
completely detached from the HTTP semantic.
The HTTP semantic should be defined on top of the framing.

> The state machine in section 5.1 is essentially a fantasy that describes an
> idealised protocol that the draft does not represent.   In my work I very
> frequently lookup the TCP/IP state machine diagram that is a great reference
> and rarely do I need to go beyond it to understand any issue I have.
> However, if developers with HTTP/2 problems try to use the state machine in
> 5.1 as a reference, they are only going to be more confused. For example,
> the ES transitions are not atomic and the closed state is described as "the
> terminal state", but is then followed by 5 paragraphs of dense complex text
> describing exceptions and how some frames received in closed state have to
> be handled!  I have posted what I think is the real state machine

I am worried about the diagram posted by Greg above. The number of
ways it could be implemented wrongly is large.
The more complex it is, the more chances of coding mistakes, attack
vectors and such.
I don't see any reason it should be so complex: closing a stream
should be similar to closing a TCP stream.

Note that there need not to be any additional semantic to
server-to-client opened streams.
The stream framing should be symmetric, and PushPromise is just a
layering of HTTP semantic on top of a server-to-client initiated
stream.
The raw SPDY framing can be used to transport something that is not
HTTP, and I have actually implemented something that transports JSON
data with no headers that is fully bidirectional.
Jetty exposes a fully symmetric SPDY-only API without binding to HTTP:
clients can open streams, and servers can open streams.

I am of the opinion that splitting the framing from the HTTP semantic
will provide many benefits, simplifying the protocol and improve HTTP
2.0.

Regarding compression of headers, I'd rather see it negotiated and off
by default. I think it is something nice to have but not absolutely
necessary.
For example in SPDY, where header compression is not that complex and
based on libraries widely available for any language (deflate), I
think the issue of header compression is nonetheless the #1 question /
problem / roadblock for SPDY implementers. HTTP 2.0 should simplify
this, not making it more complicated.

Finally, despite its limits, HTTP 1.x has been one of the most
successful protocols ever.
I would like to see HTTP 2.0 surpass it, by iteratively removing and
simplifying until there is nothing left to take away.

-- 
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 27 May 2014 03:53:21 +1000

Michael,

> *If* we can come up with a header compression scheme that does not suffer from these problems, it might be worth the added complexity in order to avoid TCP congestion window issues.  But given that…

The previous discussion that Patrick was referring to has a lot of background.

In a nutshell, he made an argument for header compression a while back (I can dig up the references if you like), where he basically showed that for a very vanilla page load, merely getting the requests out onto the wire (NOT getting any responses) would take something like 8-11 RTs, just because of the interaction between request header sizes and congestion windows. This assumes that the page has 80 assets (the average is not over 100, according to the Web archive), and request headers are around 1400 bytes (again, not uncommon).

In contrast, with compressed headers (his experiment was with gzip), you can serialise all of those requests into one RTT, perhaps even a single packet.

This is a very persuasive argument when our focus is on reducing end-user perceived latency. It’s especially persuasive when you think of the characteristics of an average mobile connection.

HPACK is not as efficient as gzip, and as we’ve said many times, our goal is NOT extremely high compression; rather, it’s safety. If we could ignore the CRIME attack, we would use gzip instead, and I don’t think we’d be having this discussion.

Hope this helps,

---

From: Werner Baumann <[email-masked]>
Date: Mon, 26 May 2014 19:58:29 +0200

Am Mon, 26 May 2014 01:00:07 -0700

Dear Mr. Belshe,

if you don't intend to argue then don't post. This list isn't for
voting.

Werner

By the way: everyone on this list already knew that you would not agree
with Poul-Henning Kamp.

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 26 May 2014 14:01:52 -0400

First, I want to let folks know that Chromium's canary channel is ready for
h2-12 interop testing as of last week (pass the "--enable-spdy4" flag, an
internal name for HTTP/2). I'll submit a PR to update the wiki. Please take
it for a spin, and let me know how it goes!

I'm a relative newcomer to HTTP/2 myself, not having heard of it (or HPACK)
prior to late January. I knew of SPDY, but hadn't worked with it. Since
then I've been working on all things SPDY & HTTP/2 within Chromium and (to
a lesser degree) Google's frontend. I've been the primary implementor of
HPACK (including Huffman), but haven't had significant input in it's design.

My opinion is that it's certainly non-trival, but it's still pretty
manageable, and it's no more complex than it needs to be. The spec is well
defined, and in practice interop with multiple parties has gone without any
gotchas surfacing. It's early still, and they may, but they haven't yet and
that's a positive datapoint.

 I have posted what I think is the real state machine
> > (

While this looks scary, I think what's actually happening here is that the
diagram is composing two independent state machines into one. The HTTP/2
stream state machine (which never transitions on CONTINUATION), with a
two-state frame parser state machine (expect continuation / don't expect
continuation). There's no reason these machines need to be composed (and
implementations I've worked with, don't).

I do agree that the close state could be firmed up. There's an third
implied, independent state machine for streams in the closed state which
bears on handling when to RST, when to ignore, when to GOAWAY with a
connection error. Perhaps this can be formalized.

> this, not making it more complicated.

Buried in this is perhaps the kernel of objections against HPACK: the
specification is too new for off-the-shelf libraries to exist. If a
requirement of SPDY were that deflate needed to be implemented from
scratch, the road to adoption would have been much longer. Unfortunately
CRIME happened, and deflate isn't an option. I suspect if good libraries
for HPACK were available, there wouldn't be nearly so much objection.

I'll note that the Chromium's license is very permissive, and Chromium's
implementation is available
here<https://code.google.com/p/chromium/codesearch#chromium/src/net/spdy/hpack_encoder.h&sq=package:chromium>.
If there's interest, I'd be happy to work with folks on packaging it as a
library.

cheers,
-johnny

---

From: James M Snell <[email-masked]>
Date: Mon, 26 May 2014 11:17:59 -0700

What would be wrong with:

1. Opting for a significantly less complicated, less aggressive compression
strategy by default; and
2. Leveraging extensibility and negotiation in the framing layer so that
HPACK can be developed and experimented with independently.

There's nothing I've seen so far that convinces me that HPACK ought to be a
normative requirement in http/2.
On May 26, 2014 10:53 AM, "Mark Nottingham" <[email-masked]> wrote:

> >> I disagree. the fundamental value of http/2 lies in mux and priority,
> and to enable both of those you need to be able to achieve a high level of
> > *If* we can come up with a header compression scheme that does not
> suffer from these problems, it might be worth the added complexity in order
> to avoid TCP congestion window issues.  But given that we are already
> facing 3.5 RTTs worth of latency just to negotiate a TLS connection I'm not
> convinced that compressing the request headers will yield a user-visible
> improvement in the speed of their web browsing experience.
>
> In a nutshell, he made an argument for header compression a while back (I
> can dig up the references if you like), where he basically showed that for
> a very vanilla page load, merely getting the requests out onto the wire
> (NOT getting any responses) would take something like 8-11 RTs, just
> because of the interaction between request header sizes and congestion
> windows. This assumes that the page has 80 assets (the average is not over
> 100, according to the Web archive), and request headers are around 1400
> bytes (again, not uncommon).
>
> In contrast, with compressed headers (his experiment was with gzip), you
> can serialise all of those requests into one RTT, perhaps even a single
> packet.
>
> This is a very persuasive argument when our focus is on reducing end-user
> perceived latency. It’s especially persuasive when you think of the
> characteristics of an average mobile connection.
>
> HPACK is not as efficient as gzip, and as we’ve said many times, our goal
> is NOT extremely high compression; rather, it’s safety. If we could ignore
> the CRIME attack, we would use gzip instead, and I don’t think we’d be
> having this discussion.
>
> Hope this helps,
>
> --

---

From: Simone Bordet <[email-masked]>
Date: Mon, 26 May 2014 20:21:26 +0200

Hi,

> HPACK is not as efficient as gzip, and as we’ve said many times, our goal is NOT extremely high compression; rather, it’s safety. If we could ignore the CRIME attack, we would use gzip instead, and…

Thanks for summing this up for a newcomer like me.
If the goal is safety, then also zero compression works.

May I ask what is plan B if a vulnerability is found in HPACK like it
happened for CRIME ?
Would that require HTTP 2.1 ?

My suggestion about negotiability of the header compression was in
this direction: simple implementations will disable it, and pay the
price Patrick showed. Better implementation will negotiate HPACK.
If HPACK is replaced or improved or found vulnerable, at least we can
run less efficient but safe until there is a review of only the
compression algorithm (and not of HTTP as a whole).

Thanks !

---

From: Daniel Stenberg <[email-masked]>
Date: Mon, 26 May 2014 22:24:17 +0200 (CEST)

On Mon, 26 May 2014, Johnny Graettinger wrote:

> specification is too new for off-the-shelf libraries to exist.

Speaking of HPACK libraries: Tatsuhiro's nghttp2 library has recently been 
getting its HPACK functionality "exposed" as an API, which is hopefully about 
to get used by the Wireshark http2 dissector...

-- 

  / daniel.haxx.se

---

From: "Martin Nilsson" <[email-masked]>
Date: Tue, 27 May 2014 00:04:54 +0200

> The previous discussion that Patrick was referring to has a lot of  

I can go further back in time and say that we considered headers to be a  
problem both to get requests ouy quickly in Opera Turbo and that headers  
sometimes created a significant overhead in the response path. We use a  
much simper compression thouhg. Both Opera Mini and Opera Turbo uses delta  
compression on header level and Mini also uses static header name table  
(kind of) and typed values (i.e. you can send a date or a size as an n  
byte integer).

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: David Krauss <[email-masked]>
Date: Tue, 27 May 2014 08:27:44 +0800

On 2014–05–26, at 10:07 PM, Tatsuhiro Tsujikawa <[email-masked]> wrote:

> ​Server and intermediaries can respond 431 if incoming headers are too large and refuse to forward to the shared backend connection.

What is the correct behavior of a proxy that gets excessive headers from a server?

What if the header stream takes too long, without being particularly large? Then the problem is not too many headers at all. Indeed, for a reverse proxy with a slow client, this is sure to happen as sure as streaming kicks in at all.

I get the sense that HTTP/2 reverse proxies can’t use multiplexing upstream. Is anyone yet testing reverse proxies?

> Receiving CONTINUATION forever is unrealistic situation.

Receiving CONTINUATION even once is unusual, if not unrealistic. The frame size was reduced simply to generate more CONTINUATIONs, so just as easily they could be all but eliminated. Yet header-streaming protocols come up as a necessary future use case. If they are not nice to have and we don’t want to support them, then a line should be drawn and a limit set.

Are there any uses *in practice on the public internet* for headers longer than 64K as serialized by HTTP/1.1?

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 27 May 2014 07:44:52 +0200

> (kind of) and typed values (i.e. you can send a date or a size as an n  

Yeah so we're back to the presentations Roy made about Waka or that I
made about our closely related proposal of typed encoding. Someone also
sent a link about the way WAP used to encode headers to reduce them. I
think that many people came to similar designs without even being in
relation because the first thing that strikes is the large header field
names for few used values, and the repetition of similar header values
between requests.

>From a technical point of view, I'd also like the header compression to
be optional, but I think that's a no-go as it will waste an RTT in the
negotiation. However we might possibly consider having a simpler variant
of HTTP/2 that does not implement header compression and which is
advertised in the TLS or upgrade handshake (eg: http/2-). That could
be useful for simple devices which don't need all the bells and whistles,
and would not cost an extra RTT.

Willy

---

From: Eliot Lear <[email-masked]>
Date: Tue, 27 May 2014 08:29:37 +0200

Hi,

> and would not cost an extra RTT.

Since there are no negotiations this is the only way to get there.  The
one nice thing about having the option is that if some problem *is*
found with HPACK browsers could switch to uncompressed without a
standards change. The real question is whether it is advisable to
continue with HTTP/2 without compression or to simply back off to 1.1. 
You can do that too, and it wouldn't require an additional code path (or
bypass), and if Patrick is correct that the benefit of /2 is tied to
compression, maybe it's the right approach.

Eliot

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 27 May 2014 08:42:32 +0200

> compression, maybe it's the right approach.

At least what I'm seeing is the possibility to consider maybe http/2
as the legacy compression-less mode that anyone should support, then
on top of that we'll be able to experiment with various compression
schemes without requiring significant upgrades. For example, nowadays,
nobody cares what TLS version or extension is supported on a given
server, things upgrade smoothly. Here we could have the same mechanism,
with a common fallback.

Considering that 1.1 would be the fallback would keep people away from
2.0 for a longer time and cast a shadow over the protocol when small
issues face up at the beginning (which is normal).

Willy

---

From: "Adrien de Croy" <[email-masked]>
Date: Tue, 27 May 2014 10:09:35 +0000

Whilst I agree that a major benefit is compression, if we don't do 
something about having to parse strings (e.g. after we decompress them) 
searching linearly for delimiters, then IMO we're missing another major 
opportunity for benefit.

Adrien

------ Original Message ------
From: "Eliot Lear" <[email-masked]>
To: "Willy Tarreau" <[email-masked]>; "Martin Nilsson" <[email-masked]>
Cc: "Michael Sweet" <[email-masked]>; "Mark Nottingham" 
<[email-masked]>; "Patrick McManus" <[email-masked]>; "James M 
Snell" <[email-masked]>; "Cory Benfield" <[email-masked]>; "Greg 
Wilkins" <[email-masked]>; "HTTP Working Group" <[email-masked]>
Sent: 27/05/2014 6:29:37 p.m.
Subject: Re: Our Schedule

---

From: Yoav Nir <[email-masked]>
Date: Tue, 27 May 2014 13:38:18 +0300

I had this idea once about replacing the encoding of headers one by one, changing most of them to compact binary fields. I even wrote a (by now long-expired) draft. The advantage of Huffman or gzip is that they work on string texts without having to design a different encoding for each header field. 

A less ambitious change would be to change the encoding of headers to something like pascal strings (length field followed by data) which would allow for faster parsing than searching for commas, quotes, etc.

Either way, that would be a nice idea for HTTP/3. I don’t think the benefit is great enough that we want to hold up HTTP/2.

Having a no-compression option in case a security flaw is discovered in HPACK is a good thing to add to HTTP/2 even at this point, but this kind of better compression?  I don’t think so.

Regards,

Yoav

On May 27, 2014, at 1:09 PM, Adrien de Croy <[email-masked]> wrote:

> 
> Whilst I agree that a major benefit is compression, if we don't do something about having to parse strings (e.g. after we decompress them) searching linearly for delimiters, then IMO we're missing another major opportunity for benefit.
> 
> Adrien
> 
> Cc: "Michael Sweet" <[email-masked]>; "Mark Nottingham" <[email-masked]>; "Patrick McManus" <[email-masked]>; "James M Snell" <[email-masked]>; "Cory Benfield" <[email-masked]>; "Greg Wilkins" <[email-masked]>; "HTTP Working Group" <[email-masked]>

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 27 May 2014 14:31:00 +0200

> searching linearly for delimiters, then IMO we're missing another major 

Yeah but only the few of us dealing with many thousands of connections
per second know the cost of parsing strings unfortunately. I've long
abandonned this battle considering that other processings were added
at some points with several orders of magnitude higher costs (eg:
gzipping of headers), and fortunately replaced.

Willy

---

From: Mike Bishop <[email-masked]>
Date: Tue, 27 May 2014 14:17:49 +0000

The obvious issue is that you actually need the most aggressive compression on the first packet, while the CWND is small.  With negotiation, you wouldn’t get to move to the better compression until at least an RTT later, when you have more window to send less-efficiently-compressed requests.

Not that you can’t do something here, but I think that’s the hurdle to overcome.

From: James M Snell [mailto:[email-masked]]
Sent: Monday, May 26, 2014 11:18 AM
To: Mark Nottingham
Cc: Patrick McManus; Cory Benfield; Greg Wilkins; HTTP Working Group; Michael Sweet
Subject: Re: Our Schedule

1. Opting for a significantly less complicated, less aggressive compression strategy by default; and
2. Leveraging extensibility and negotiation in the framing layer so that HPACK can be developed and experimented with independently.

There's nothing I've seen so far that convinces me that HPACK ought to be a normative requirement in http/2.
On May 26, 2014 10:53 AM, "Mark Nottingham" <[email-masked]<mailto:[email-masked]>> wrote:
Michael,

On 27 May 2014, at 2:35 am, Michael Sweet <[email-masked]<mailto:[email-masked]>> wrote:

> Patrick,
>
> On May 26, 2014, at 10:45 AM, Patrick McManus <[email-masked]<mailto:[email-masked]>> wrote:
>> ...

Hope this helps,

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 27 May 2014 23:49:48 +0900

On Tue, May 27, 2014 at 9:27 AM, David Krauss <[email-masked]> wrote:

> large and refuse to forward to the shared backend connection.
>
>
​There are several actions proxy can take:
1) Ignore excessive headers
2) RST_STREAM
3) GOAWAY (if one header is too large to handle and keep in header table)

In framing protocol wise, all are correct behavior.​  Proxy has right to
protect its own.

> What if the header stream takes too long, without being particularly
> large? Then the problem is not too many headers at all. Indeed, for a
> reverse proxy with a slow client, this is sure to happen as sure as
> streaming kicks in at all.
>
>
​This is not limited to HEADERS.  Just send 1 byte of first header of any
frame and pause.​
​This is good candidate for the timeout, isn't it?​

> I get the sense that HTTP/2 reverse proxies can’t use multiplexing
> upstream. Is anyone yet testing reverse proxies?
>
> Receiving CONTINUATION even once is unusual, if not unrealistic. The frame
> size was reduced simply to generate more CONTINUATIONs, so just as easily
> they could be all but eliminated. Yet header-streaming protocols come up as
> a necessary future use case. If they are not nice to have and we don’t want
> to support them, then a line should be drawn and a limit set.
>
> Are there any uses **in practice on the public internet** for headers
> longer than 64K as serialized by HTTP/1.1?
>
>
​I don't have actual statistics of length of the header.  But server
software have its own limit of header buffers.
For example, HAproxy uses 8K buffer for entire request headers by default
and its manual says it is not recommended to change the value, so limit is
somewhere near it?

Best regards,
Tatsuhiro Tsujikawa

---

From: David Krauss <[email-masked]>
Date: Wed, 28 May 2014 10:21:46 +0800

> 3) GOAWAY (if one header is too large to handle and keep in header table)

Off topic a bit, but if one header is too large to fit in the header table, the current spec of HPACK requires the table to be cleared. I think the table should be left unchanged; is this an erratum?

Anyway, no need to GOAWAY in that case.

Yes, there would be a RST_STREAM upstream and downstream, and that would suck. What would even be an appropriate error code? Internal error?

> 
> In framing protocol wise, all are correct behavior.​  Proxy has right to protect its own.
>  
> ​This is not limited to HEADERS.  Just send 1 byte of first header of any frame and pause.​

I’m talking about the connection between the reverse proxy and the server. There should be only one connection there, representing all the clients assigned to the server. The reverse proxy is trusted, and it can always send complete data frames. However it is still at the mercy of the client as for complete header blocks, under a streaming policy.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 28 May 2014 21:21:40 +0900

On Wed, May 28, 2014 at 11:21 AM, David Krauss <[email-masked]> wrote:

> Off topic a bit, but if one header is too large to fit in the header
> table, the current spec of HPACK requires the table to be cleared. I think
> the table should be left unchanged; is this an erratum?
>
​Yeah, you are right, GOAWAY can be avoidable in this case.​
​If the incoming header is very very long, GOAWAY may be an option.​

> Yes, there would be a RST_STREAM upstream and downstream, and that would
> suck. What would even be an appropriate error code? Internal error?
>
>
​ENHANCE_YOU_CALM is suggested in the another thread.

Best regards,
Tatsuhiro Tsujikawa​

> I’m talking about the connection between the reverse proxy and the server.
> There should be only one connection there, representing all the clients
> assigned to the server. The reverse proxy is trusted, and it can always
> send complete data frames. However it is still at the mercy of the client
> as for complete header blocks, under a streaming policy.
>
>

---

From: David Krauss <[email-masked]>
Date: Wed, 28 May 2014 23:24:59 +0800

> ​ENHANCE_YOU_CALM is suggested in the another thread.

The client sends an innocuous, small request to the origin via a proxy. The origin sends too many headers back. So, the proxy tells the client to ENHANCE_YOUR_CALM?

> I’m talking about the connection between the reverse proxy and the server. There should be only one connection there, representing all the clients assigned to the server. The reverse proxy is trust…

If nothing can be done about this, HTTP/2 reverse proxies would seem to be bound to multi-connection topology with respect to the upstream servers. Perhaps not a step backwards, but it’s still a compromise.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Thu, 29 May 2014 00:52:18 +0900

On Thu, May 29, 2014 at 12:24 AM, David Krauss <[email-masked]> wrote:

> The client sends an innocuous, small request to the origin via a proxy.
> The origin sends too many headers back. So, the proxy tells the client to
> ENHANCE_YOUR_CALM?
>
>
​I thought that it goes from proxy to origin.  I agree that it is not
appropriate to the client side.​

>> I’m talking about the connection between the reverse proxy and the
>> server. There should be only one connection there, representing all the
>> clients assigned to the server. The reverse proxy is trusted, and it can
>> always send complete data frames. However it is still at the mercy of the
>> client as for complete header blocks, under a streaming policy.
>>
>
> If nothing can be done about this, HTTP/2 reverse proxies would seem to be
> bound to multi-connection topology with respect to the upstream servers.
> Perhaps not a step backwards, but it’s still a compromise.
>
>
​Our current implementation buffers all incoming HEADERS in proxy and does
not forward it in streaming manner.
If if incoming headers are too large to buffer,​ we currently just emits
RST_STREAM to both ends.  I think we can do more elegantly, but this is the
way I do ATM.  For slow client/server, we have timeout for each connection,
but no timeout for each stream at the moment.
​Does this answer your question?

​Best regards,
Tatsuhiro Tsujikawa

---

From: James M Snell <[email-masked]>
Date: Wed, 28 May 2014 08:59:06 -0700

This thread is becoming increasingly impossible to follow. As per the
original point regarding the schedule: my preference would be to not rush
or fast track a WGLC but rather let the spec sit and simmer for a bit while
more implementors catch up and become able to provide  evidence that http/2
as it is currently defined is satisfactory.
On May 27, 2014 7:17 AM, "Mike Bishop" <[email-masked]> wrote:

>  The obvious issue is that you actually need the most aggressive
> compression on the first packet, while the CWND is small.  With
> negotiation, you wouldn’t get to move to the better compression until at
> least an RTT later, when you have more window to send
> less-efficiently-compressed requests.
>
>
>
> Not that you can’t do *something* here, but I think that’s the hurdle to
> overcome.
>
>
>
> *From:* James M Snell [mailto:[email-masked]]
> *Sent:* Monday, May 26, 2014 11:18 AM
> *To:* Mark Nottingham
> *Cc:* Patrick McManus; Cory Benfield; Greg Wilkins; HTTP Working Group;
> Michael Sweet
> *Subject:* Re: Our Schedule
>
>
>

---

From: Jason Greene <[email-masked]>
Date: Wed, 28 May 2014 13:35:39 -0500

> gzipping of headers), and fortunately replaced.

Why is such a stark trade-off necessary? Why not make HPACK optional and define a compact record structure for headers, with numerical constants for all standard headers and standard values? 

It would be nice if the protocol was optimal for the server as well as the client.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat

---

From: Roberto Peon <[email-masked]>
Date: Wed, 28 May 2014 11:51:33 -0700

That is essentially what HPACK is, if one sets max-state to zero.
HPACK does length-delimited key-value pairs, with numerical constants for
standard headers and values.
-=R

On Wed, May 28, 2014 at 11:35 AM, Jason Greene <[email-masked]>wrote:

> Why is such a stark trade-off necessary? Why not make HPACK optional and
> define a compact record structure for headers, with numerical constants for
> all standard headers and standard values?
>

---

From: Jason Greene <[email-masked]>
Date: Wed, 28 May 2014 14:10:53 -0500

On May 28, 2014, at 1:51 PM, Roberto Peon <[email-masked]> wrote:

> HPACK does length-delimited key-value pairs, with numerical constants for standard headers and values.
> -=R

Ah yes. It looks very efficient actually. I should have verified the premise before commenting.

-Jason