Subject: #541: CONTINUATION

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 21:12:12 +1000

<https://github.com/http2/http2-spec/issues/541>

There’s been strong pushback on the current design of CONTINUATION from some interested parties, and a few implementers. Despite the fact that this design is the result of multiple meetings demonstrating strong consensus, and the fact that we have a schedule-focused charter, this issue deserves a good hearing.

I think everyone now has an idea of the issues and trade-offs involved, as well as the potential end-games. We also helpfully have a few proposals on how to move forward:

0) the status quo

1) <https://github.com/http2/http2-spec/pull/544> and variants thereof (e.g., not including CONTINUATION in flow control; alternative syntaxes)

2) limiting header sizes to 16K (HPACK’d) in HTTP/2, as per PHK’s suggestion

There’s also another implicit option; 

3) We can’t know until we get substantial interop and deployment experience with draft-13.

I’d like to ask the implementers (as identified on the CC: line) what their preferences are, and what they can’t live with. If there’s another option, please say so, but only if it’s materially different, and you believe it has a chance of achieving consensus. 

To be clear, if you don’t say that you can’t live with something, it means that it’s an acceptable outcome. If you do, be prepared to back up such a strong stance with an equally strong argument.

Note that this is input to help determine consensus, not a vote.

Thanks,

P.S. Please keep in mind that (3) is not “wait until September, then decide it’s too late.” Achieving a reasonable consensus now is relatively pain-free, if possible; deadlocking right before we (want to) ship is something I want to avoid.

P.P.S. To anticipate some responses, a generic “jumbo frame” is off the table for this discussion; doing so doesn’t appear to have broad support, and there are strong arguments against it.

--
Mark Nottingham   http://www.mnot.net/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Jul 2014 21:30:20 +1000

Mark,

thanks for re-opening this issue with a focus on what we can live with.

For jetty's part we can  live with 0) or 3),  but are having increasing
concerns about it with the suggestions like Will's of clients deliberately
trying to break implementations that choose not to support continuations.
If that really is a meme, then we can't live with 0) and probably not 3).

I guess we need to put option 1) into the same group as 0) and 3). If we
can not implement continuation frames then we don't care if they are
compressed or not - we still will not implement a mechanism that is not
used by 99.99% of our users.

2) we obviously can live with.  However it is not my preference to not
support large headers.  I wish to support them with a seamless
implementation that is just a configuration change - ie jumbo frames.  A
large header extension is also OK.

Our preference is jumbo frames, as I believe they support large headers,
optionally support large data frames for the issue Willy brought up; and
they are no more complex and a lot less ugly than continuations.

But as jumbo frames have been ruled out as an option, we can live with 2).
We can live with 0), 1) and 3), so long as it does not become a crime (of
blackmail or otherwise) to not support CONTINUATIONs.

cheers

PS. If jumbo frames were to be at least considered I would undertake to
write up a full proposal for them.

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: <[email-masked]>
Date: Wed, 2 Jul 2014 11:29:30 +0000

What counts as an implementer?  Is there an IETF rule that only implementers count toward consensus?

-keith

On Wednesday,02 July 2014 13:12, [email-masked] wrote:

There's been strong pushback on the current design of CONTINUATION from some interested parties, and a few implementers. Despite the fact that this design is the result of multiple meetings demonstrating strong consensus, and the fact that we have a schedule-focused charter, this issue deserves a good hearing.

0) the status quo

2) limiting header sizes to 16K (HPACK'd) in HTTP/2, as per PHK's suggestion

There's also another implicit option;

3) We can't know until we get substantial interop and deployment experience with draft-13.

I'd like to ask the implementers (as identified on the CC: line) what their preferences are, and what they can't live with. If there's another option, please say so, but only if it's materially different, and you believe it has a chance of achieving consensus.

To be clear, if you don't say that you can't live with something, it means that it's an acceptable outcome. If you do, be prepared to back up such a strong stance with an equally strong argument.

Thanks,

P.S. Please keep in mind that (3) is not "wait until September, then decide it's too late." Achieving a reasonable consensus now is relatively pain-free, if possible; deadlocking right before we (want to) ship is something I want to avoid.

P.P.S. To anticipate some responses, a generic "jumbo frame" is off the table for this discussion; doing so doesn't appear to have broad support, and there are strong arguments against it.

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Shigeki Ohtsu <[email-masked]>
Date: Wed, 02 Jul 2014 20:31:33 +0900

My preference is

0) the status quo

and I can't live with 1, 2 and 3 as already noted in

http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1779.html

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 21:32:28 +1000

> thanks for re-opening this issue with a focus on what we can live with.

To be clear — 541 wasn’t closed...

> For jetty's part we can  live with 0) or 3),  but are having increasing concerns about it with the suggestions like Will's of clients deliberately trying to break implementations that choose not to support continuations.  If that really is a meme, then we can't live with 0) and probably not 3).
> 
> I guess we need to put option 1) into the same group as 0) and 3). If we can not implement continuation frames then we don't care if they are compressed or not - we still will not implement a mechanism that is not used by 99.99% of our users.
> 
> 2) we obviously can live with.  However it is not my preference to not support large headers.  I wish to support them with a seamless implementation that is just a configuration change - ie jumbo frames.  A large header extension is also OK.   
> 
> Our preference is jumbo frames, as I believe they support large headers, optionally support large data frames for the issue Willy brought up; and they are no more complex and a lot less ugly than continuations.  
> 
> But as jumbo frames have been ruled out as an option, we can live with 2).  We can live with 0), 1) and 3), so long as it does not become a crime (of blackmail or otherwise) to not support CONTINUATIONs.
> 
> cheers
> 
> PS. If jumbo frames were to be at least considered I would undertake to write up a full proposal for them.

Thanks.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 21:34:55 +1000

Keith,

On 2 Jul 2014, at 9:29 pm, <[email-masked]> <[email-masked]> wrote:

> What counts as an implementer?

Someone who’s listed on <https://github.com/http2/http2-spec/wiki/Implementations> with a reasonably current version.

> Is there an IETF rule that only implementers count toward consensus?

No… as I very clearly said, “Note that this is input to help determine consensus, not a vote.”

Specifically, we want rough consensus and running code; this question is trying to determine where the running code will be.

Kind regards,

P.S. I probably should have done a reply-to on that message; please be kind to your fellow WG members and strip the CC. Thanks.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 02 Jul 2014 12:29:00 +0000

>2) limiting header sizes to 16K (HPACK'd) in HTTP/2, as per PHK's 

Personally I'd prefer this expression:

	2) All headers, after (any) compression, must fit in a single
	   HEADERS frame.

This expression allows people to negotiate a jumbo-frame extension
and escape the ill-adviced 16383 bytes hard limit.

This is going to be Varnish take on any variant of CONTINUATION
as they are currently specified:  We simply won't touch them,
and sever any connection that tries to use them.

(I think it would be wise to subject HEADERS to flow-control,
but am open for other anti-DoS mechanisms on this point.)

>experience with draft-13.

I don't think we need to floor it on the freeway to see that painting
the windshield was a bad idea, and punting now is just a tacit
acceptance of 0) by inaction.

Poul-Henning

PS:
    Yes, I realize that not having wasted a lot of time implementing
    a protocol I think sucks in too many important ways disqualifies
    me as "a implementor" using Marks strict criteria but the code
    that does not get written due to bad standardization should
    also be listened to.

PPS:
    I'm looking for co-authors for a jumboframe extension draft.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 22:33:18 +1000

> But as jumbo frames have been ruled out as an option, we can live with 2).  We can live with 0), 1) and 3), so long as it does not become a crime (of blackmail or otherwise) to not support CONTINUA…

I ruled out jumbos because there was poor support for them when we resolved that issue; if people have a change of heart based on this discussion, it’s worth knowing. I just want to keep this discussion as manageable as possible…

Cheers,

---

From: Cory Benfield <[email-masked]>
Date: Wed, 2 Jul 2014 14:39:02 +0100

Mark,

> I’d like to ask the implementers (as identified on the CC: line) what their preferences are, and what they can’t live with. If there’s another option, please say so, but only if it’s materially dif…

Speaking strictly for hyper, I can live with any of the above options,
assuming that everyone else can live with them. What I cannot live
with is any proposal that ends up with having some intermediaries or
servers reject some headers without me being able to discover this
ahead of time.

As an idealist I'd like to be able to ban large header sizes outright,
but I believe that action to be outside our charter, so I suspect that
2) is unacceptable. I would absolutely want to put it on the table
if/when HTTP/3 gets discussed.

If we settle on leaving CONTINUATION in place but some intermediaries
continue to refuse to use them, I'll consider writing an extension for
negotiating their use. I'd rather a server/intermediary told me "I
will kill this connection if you send this frame" (or, more likely, "I
will kill this connection if you send more than this many bytes of
headers"). I appreciate that the CONTINUATION frame is not intended to
be optional, but for me pragmatism rules over idealism.

Cory

---

From: Yoav Nir <[email-masked]>
Date: Wed, 2 Jul 2014 16:56:55 +0300

Not an implementer of HTTP myself - those are in the other building, but I do meet with them regularly (and no, we won’t have something ready in the relevant timeframe)

I don’t believe that the interop and deployment experience from draft-13 will provide new information for this issue. 

I’d also like to point out that #2 has an issue with middleboxes. Middleboxes futz with headers. So a client or server may send headers that are less than 16K when HPACK’d, but after futzing and re-HPACK-ing, those can exceed 16K. So a client sends a HEADERS frame that seems entirely valid, only for the stream to be terminated by the proxy because of the excessive size of the headers.

I would have been fine with jumbo-frames even if they’re only for headers (although I’d like them to be for everything), even if they were a negotiated option. But absent jumbo-frames on the menu, I’d go with #0.

Yoav

---

From: Jason Greene <[email-masked]>
Date: Wed, 2 Jul 2014 09:18:33 -0500

On Jul 2, 2014, at 6:30 AM, Greg Wilkins <[email-masked]> wrote:

> For jetty's part we can  live with 0) or 3),  but are having increasing concerns about it with the suggestions like Will's of clients deliberately trying to break implementations that choose not to support continuations.  

Keep in mind that to be compliant 0 requires that you support continuations you can’t opt out of them *even* if you have a 16KB length limit.

1) The spec allows a peer to send lots of tiny CONTINUATION frames (and it looks like even empty are allowed). Chunking intermediaries has come up as a use case for this.

2) Even if you reject headers due to size, a limitless number of CONTINUATION frames must still be processed to keep the connection alive. 

That said you are free to GOAWAY in DOS scenarios; however, it still stands that implementations can not opt-out and remain compliant. Even if someone promises they won’t create a walled garden, there are no guarantees.

I was thinking of proposing an option 4, where a setting is added that specifies a total header frame(s) length limit, and a default limit of 16KB. This eliminates the HOL problems by default, and lets venders choose how much of one they are willing to accept to allow some bigger frames in. I’d prefer fixing continuations or jumbo frames, but at least this keeps everyone who is against from being non-compliant, and allows intermediaries to avoid relaying stuff that will only be rejected.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat

---

From: Mike Bishop <[email-masked]>
Date: Wed, 2 Jul 2014 17:07:23 +0000

I like this, especially in combination with Willy's restriction of 0).  CONTINUATION is only valid if the preceding frame was max_size (so it really is a continuation and not an arbitrary fragment), and there are optional settings that let a server advertise its limits.  I dislike the unpredictability of HTTP/1.1 in this regard, and wouldn't mind an extension or option in the spec to make things more transparent.

I would suggest a few refinements, though:
  - HPACK is effective.  We, at least, will be bounding on the *uncompressed* sizes, not the size that goes into the frame.

  - We've been conflating total header set size and maximum size of a single header; both are commonly-used caps, but they're different things.  We need op-codes to be able to cross frame boundaries, because individual headers can be larger than 16KB; we need CONTINUATION both for that, *and* because the total header set can be larger than 16KB even if no individual header is.  We should advertise both, if we're trying to fix the ambiguity.

  - There have been concerns expressed about being too specific about your limits, lest people try to skate too close to them.  Perhaps there should be a way to express undefined if someone wants legacy behavior for some reason.

-----Original Message-----
From: Jason Greene [mailto:[email-masked]] 
Sent: Wednesday, July 2, 2014 7:19 AM
To: Greg Wilkins
Cc: Mark Nottingham; HTTP Working Group; Tatsuhiro Tsujikawa; Mike Bishop; Gábor Molnár; Patrick McManus; Shigeki Ohtsu; Ludin, Stephen; Johnny Graettinger; Hasan Khalil; Jeff Pinner; [email-masked]; Adrian Cole; [email-masked]; Cory Benfield; Daniel Stenberg; Flack, Martin; [email-masked]; Jeroen de Borst
Subject: Re: #541: CONTINUATION

> For jetty's part we can  live with 0) or 3),  but are having increasing concerns about it with the suggestions like Will's of clients deliberately trying to break implementations that choose not to…

Keep in mind that to be compliant 0 requires that you support continuations you can't opt out of them *even* if you have a 16KB length limit.

That said you are free to GOAWAY in DOS scenarios; however, it still stands that implementations can not opt-out and remain compliant. Even if someone promises they won't create a walled garden, there are no guarantees.

I was thinking of proposing an option 4, where a setting is added that specifies a total header frame(s) length limit, and a default limit of 16KB. This eliminates the HOL problems by default, and lets venders choose how much of one they are willing to accept to allow some bigger frames in. I'd prefer fixing continuations or jumbo frames, but at least this keeps everyone who is against from being non-compliant, and allows intermediaries to avoid relaying stuff that will only be rejected.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect JBoss, a division of Red Hat

---

From: Nicholas Hurley <[email-masked]>
Date: Wed, 2 Jul 2014 10:54:02 -0700

[Trimmed CC]

Speaking from the perspective of working on both Firefox (client) and (more
recently) node-http2 (server) implementations, I strongly prefer keeping
the status quo. I have seen no arguments at this point that convince me
that a relatively large change to how we compress and frame headers is in
any way required. We're talking about handling (iirc) 0.02% of the cases
here, and I frankly think it makes sense to say that if you've received
more compressed (or uncompressed!) headers than you're willing to deal
with, then just send a RST_STREAM and process and discard any remaining
HEADERS/CONTINUATION frames on that stream (or send a GOAWAY if you feel
you're in a DoS situation). It seems to me that a robust implementation
will already have a facility to process and discard HPACK data anyway, to
handle other situations where you may decide while receiving headers that
you don't want the stream.

I *could* work with stateless CONTINUATION frames in both of these
implementations, though it would still require rolling back the state of at
least one encoded header (once it's determined that the encoding would
cause a crossing of the 16k boundary). It also breaks the separation of
concerns between HPACK and the framing layer in a way that I find
*significantly* less than appealing. As such, even though I won't burn
things down if this is the option the WG comes to consensus on, I won't go
down without a fight :) To be clear, there is nothing in my server-side
implementation experience that has indicated to me that even this
alteration is necessary, or even a nice-to-have.

None of the other options are acceptable to me.
--
Peace,
  -Nick

On Wed, Jul 2, 2014 at 4:12 AM, Mark Nottingham <[email-masked]> wrote:

> There’s been strong pushback on the current design of CONTINUATION from
> some interested parties, and a few implementers. Despite the fact that this
> design is the result of multiple meetings demonstrating strong consensus,
> and the fact that we have a schedule-focused charter, this issue deserves a
> good hearing.
>
> I think everyone now has an idea of the issues and trade-offs involved, as
> well as the potential end-games. We also helpfully have a few proposals on
> how to move forward:
>
> 0) the status quo
>
> 1) <https://github.com/http2/http2-spec/pull/544> and variants thereof
> (e.g., not including CONTINUATION in flow control; alternative syntaxes)
>
> I’d like to ask the implementers (as identified on the CC: line) what
> their preferences are, and what they can’t live with. If there’s another
> option, please say so, but only if it’s materially different, and you
> believe it has a chance of achieving consensus.
>
> To be clear, if you don’t say that you can’t live with something, it means
> that it’s an acceptable outcome. If you do, be prepared to back up such a
> strong stance with an equally strong argument.
>
> P.S. Please keep in mind that (3) is not “wait until September, then
> decide it’s too late.” Achieving a reasonable consensus now is relatively
> pain-free, if possible; deadlocking right before we (want to) ship is
> something I want to avoid.
>
> P.P.S. To anticipate some responses, a generic “jumbo frame” is off the
> table for this discussion; doing so doesn’t appear to have broad support,
> and there are strong arguments against it.
>
>
> --

---

From: Jason Greene <[email-masked]>
Date: Wed, 2 Jul 2014 13:35:53 -0500

Removing CC entries as requested by Mark N.

> I like this, especially in combination with Willy's restriction of 0).  CONTINUATION is only valid if the preceding frame was max_size (so it really is a continuation and not an arbitrary fragment)…

Thanks for the feedback. If I understand you correctly, you are suggesting this be optional? I was suggesting something more along the lines of SETTINGS_HEADER_TABLE_SIZE, that would be called something like SETTINGS_MAX_HEADER_BLOCK_SIZE, which would default to 16KB. If you combined it with Willy’s proposal that would mean no CONTINUATIONS by default. The big CON of this possible proposal is that the .02% case would have to wait for a SETTINGS ACK before it can send a mega header. On the other hand, not wanting to improve the QOS of .02% seems to be a popular sentiment. The main pro I am after is mitigating the HOL blocking aspects of the current draft, and some form of reasonable default restriction would do that. 

>  - HPACK is effective.  We, at least, will be bounding on the *uncompressed* sizes, not the size that goes into the frame.

That’s a good point, and is perhaps better information for the client. The primary goal I had in mind though was balancing streams. Once the full header block (Header + N continuations) exceeds the max 16kb frame size you start introducing a fairness problem. So it seemed simpler to base it on that. However, input length is just as effective at controlling that. 

>  - We've been conflating total header set size and maximum size of a single header; both are commonly-used caps, but they're different things.  We need op-codes to be able to cross frame boundaries…

OK I can see that being useful to individual end points.

>  - There have been concerns expressed about being too specific about your limits, lest people try to skate too close to them.  Perhaps there should be a way to express undefined if someone wants le…

I have no qualms with that.

---

From: Daniel Stenberg <[email-masked]>
Date: Wed, 2 Jul 2014 23:15:12 +0200 (CEST)

On Wed, 2 Jul 2014, Mark Nottingham wrote:

> I’d like to ask the implementers (as identified on the CC: line) what their 
> preferences are, and what they can’t live with.

I can live with either way. My order of preference is 0, 1, 2. But yes, we 
will of course know more at (3).

-- 

  / daniel.haxx.se

---

From: Adrian Cole <[email-masked]>
Date: Wed, 2 Jul 2014 15:29:41 -0700

Speaking from okhttp experience, I'd prefer 2, jumbo frame, or 0 (if
we can't  agree on 2 or jumbo).

-A

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Thu, 3 Jul 2014 09:55:59 +0900

>
> I think everyone now has an idea of the issues and trade-offs involved,
as well as the potential end-games. We also helpfully have a few proposals
>
> To be clear, if you don’t say that you can’t live with something, it
means that it’s an acceptable outcome. If you do, be prepared to back up
such a strong stance with an equally strong argument.
>
>
> --
> Mark Nottingham   http://www.mnot.net
>
>

My preference is (0).
(1) adds more complexity just for 0.02% cases.
h2 should support > 16k headers so (2) is not an option.
(3) may be a candidate but if we can agree now then it is better.

So if 0.02% requests really causes a problem, specifically in coalesce
scenario, I'd like to suggest that we can add minimum size of compressed
header payload size that must be supported.  I choose compressed size
because it can be easily computed by payload length.  The default size is
64k.    Receiver can freely respond with 413 within this limit but it has
to burn HPACK to decompress headers (and discard them).  It can terminate
connection when it encoders more than this limit.
Sender should not send header block more than 64k in total in compressed
size.  It is possible to roughly estimate upper bound of compressed size
without HPACK (encode all new name literal, possible encodong context
updates, reference set toggle off at the end).

Best regards,
Tatsuhiro Tsujikawa

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 3 Jul 2014 11:16:16 +1000

On 3 Jul 2014, at 10:55 am, Tatsuhiro Tsujikawa <[email-masked]> wrote:

> So if 0.02% requests really causes a problem, specifically in coalesce scenario, I'd like to suggest that we can add minimum size of compressed header payload size that must be supported.  I choose compressed size because it can be easily computed by payload length.  The default size is 64k.    Receiver can freely respond with 413 within this limit but it has to burn HPACK to decompress headers (and discard them).  It can terminate connection when it encoders more than this limit.
> Sender should not send header block more than 64k in total in compressed size.  It is possible to roughly estimate upper bound of compressed size without HPACK (encode all new name literal, possible encodong context updates, reference set toggle off at the end).

Thank you.

Any thoughts about the subsequent proposal #4?

http://www.w3.org/mid/[email-masked]

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 3 Jul 2014 11:16:09 +1000

Let’s call this general direction #4; it seems worth pursuing.

Thanks,

---

From: Mike Bishop <[email-masked]>
Date: Thu, 3 Jul 2014 01:20:04 +0000

What is "this" in your question?  I get the MAX_HEADER_BLOCK_SIZE proposal, and I'm suggesting that:
	- It refers to uncompressed header size, not compressed
	- The default is 16 KB; implementations that like that value need not send it.  (In fact, that's our out-of-the-box default, too -- it takes explicit configuration to increase the acceptable header size if your site needs it.)
	- CONTINUATION be constrained to following a max-size HEADERS, PUSH_PROMISE, or CONTINUATION frame; implementations that use the default value of MAX_HEADER_BLOCK_SIZE can reject a CONTINUATION frame from the client for violating one of several constraints.

Yes, this means waiting one RTT for large headers -- that's probably okay, since if our primary use-case is authentication, you need to wait one RTT to get challenged for auth to begin with.  (If you know in advance you need to auth, you may also know that the server supports large headers.)

The real question with such an extension is the reverse direction -- does the client also advertise their maximum response header size?  What should a server do if the request a client sent produces a response the client isn't going to accept due to header size?

-----Original Message-----
From: Jason Greene [mailto:[email-masked]] 
Sent: Wednesday, July 2, 2014 11:36 AM
To: Mike Bishop
Cc: Greg Wilkins; Mark Nottingham; HTTP Working Group
Subject: Re: #541: CONTINUATION

> I like this, especially in combination with Willy's restriction of 0).  CONTINUATION is only valid if the preceding frame was max_size (so it really is a continuation and not an arbitrary fragment)…

Thanks for the feedback. If I understand you correctly, you are suggesting this be optional? I was suggesting something more along the lines of SETTINGS_HEADER_TABLE_SIZE, that would be called something like SETTINGS_MAX_HEADER_BLOCK_SIZE, which would default to 16KB. If you combined it with Willy's proposal that would mean no CONTINUATIONS by default. The big CON of this possible proposal is that the .02% case would have to wait for a SETTINGS ACK before it can send a mega header. On the other hand, not wanting to improve the QOS of .02% seems to be a popular sentiment. The main pro I am after is mitigating the HOL blocking aspects of the current draft, and some form of reasonable default restriction would do that. 

>  - HPACK is effective.  We, at least, will be bounding on the *uncompressed* sizes, not the size that goes into the frame.

That's a good point, and is perhaps better information for the client. The primary goal I had in mind though was balancing streams. Once the full header block (Header + N continuations) exceeds the max 16kb frame size you start introducing a fairness problem. So it seemed simpler to base it on that. However, input length is just as effective at controlling that. 

> Cc: Mark Nottingham; HTTP Working Group; Tatsuhiro Tsujikawa; Mike 
> Bishop; Gábor Molnár; Patrick McManus; Shigeki Ohtsu; Ludin, Stephen; 
> Johnny Graettinger; Hasan Khalil; Jeff Pinner; 
> [email-masked]; Adrian Cole; 
> [email-masked]; Cory Benfield; Daniel Stenberg; 
> Flack, Martin; [email-masked]; Jeroen de Borst

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 03 Jul 2014 18:29:31 +1200

>     I'm looking for co-authors for a jumboframe extension draft.

Do you really need them? if you publish a sensible draft I expect 
several of us not liking CONTINUATION would implement.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 3 Jul 2014 16:36:15 +1000

On 3 Jul 2014, at 4:29 pm, Amos Jeffries <[email-masked]> wrote:

> Do you really need them? if you publish a sensible draft I expect several of us not liking CONTINUATION would implement.

Would you (or any of the other Jumbo advocates) mind explaining why on its own it’s better than CONTINUATION, beyond syntactic sugar?

AIUI you have to buffer and decode the entire header set in either case; is there some other aspect where it’s significantly better?

Apologies if that’s been covered already, but it’s a long conversation...

Regards,

---

From: "Walter H." <[email-masked]>
Date: Thu, 3 Jul 2014 08:42:31 +0200

>     I'm looking for co-authors for a jumboframe extension draft.

this is a mistake by design, because, jumbo frames are an extension to
Ethernet, and this is some layer deeper and has nothing to do with http,
nor with tcp/ip itself;

---

From: Yoav Nir <[email-masked]>
Date: Thu, 3 Jul 2014 10:07:22 +0300

> Would you (or any of the other Jumbo advocates) mind explaining why on its own it’s better than CONTINUATION, beyond syntactic sugar?

(I probably don’t really count as an advocate, but I’ll give it a try anyway)

I think it mostly is syntactic sugar, but it makes for a more sane layering of software. You’re going to have a layer that deals with TCP (that does the actual SkRead). The next layer - let’s call it the framing layer - will read a header (with length) followed by the body of the frame. Then, depending on the type of frame, it will move the data to some handler (data handler, header handler). 

With jumboframes (and more importantly - no continuation) the “header handler” never needs to buffer - it can just handle the whole thing at once. A requirement that CONTINUATION frames immediately follow HEADERS/PP frames with no intervening frames achieves a close enough goal, and that is already in the draft.

So I don’t think that’s a huge issue, as the data handler has to live with continuations (more DATA frames) anyways.

Jumboframes for data are a different issue. They have a marginal increase in efficiency (in that you don’t have to send an 8-byte header every 16KB). It’s tempting to always send a data jumboframe that contains as much data as is either left or allowed by flow control, but I suspect that would have a really bad interaction with the flow control.

Yoav

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 3 Jul 2014 17:11:56 +1000

On 3 July 2014 16:36, Mark Nottingham <[email-masked]> wrote:

> Would you (or any of the other Jumbo advocates) mind explaining why on its
> own it’s better than CONTINUATION, beyond syntactic sugar?
>
> AIUI you have to buffer and decode the entire header set in either case;
> is there some other aspect where it’s significantly better?

I've been very keen to say that Continuations ARE Jumbo frames, but I do
see some key differences that go beyond syntactic sugar:

   - The max acceptable size will be declared in a SETTING, allowing for it
   to be know in advance
   - max frame size can be adjusted down for low resource impls.
   - A true jumbo frame has its total size in the first frame, so an impl
   can immediately know if it is acceptable or not. With continuations, you
   might have to process 5 frames to reach a 64KB limit, only then to find out
   that their is a 6th frame and you have to throw away all that work done.
   - The code path for handling large or small frames will be pretty much
   the same for jumbo frames, with the exception of the code that determines
   the frame length.   With continuations, there is a significantly different
   code path that will be executed only for headers that are larger than
   20-something KB.   This is a code base that will be rarely used and thus
   probably not well tested.  Here be dragons!
   - Jumbo frames can also be applied to data frames (IF both the endpoints
   so desire).   This can be done to either simply tune the frame size to the
   most efficient size for the network (eg the fast hop between a load
   balancer and application server maybe able to have a much larger frame size
   and still have good multiplexing), or it may even be used to allow a single
   frame to send very large content.
   - Closing stuff down is just really hard.  No its harder than that!  I
   mean really really difficult!   SSL handshakes, half closed TCP
   connections, race conditions, blah blah blah - it is just really really
   really difficult to write robust code that always closes neatly, without
   double/triple work and does not leak any resources.    Any simplification
   in this part of the code will be really really really valuable.
   - Jumbo frames means that the END_STREAM flag is set on the last frame
   of a stream.   It is a real WTF moment for anybody looking at the spec when
   they see CONTINUATIONS can be sent after END_STREAM.  The confusion this
   causes should not be underestimated and reading this list is a good
   indication of the communication problems that will result.
   - A finally - CONTINUATIONS are jumbo frames.     If it quacks like a
   duck....

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 03 Jul 2014 07:43:17 +0000

>Ethernet, and this is some layer deeper and has nothing to do with http,

Uhm, we're talking about HTTP/2 frames, not ethernet frames.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 03 Jul 2014 07:54:51 +0000

In message <[email-masked]>, Mark Nottingham writes:
>Would you (or any of the other Jumbo advocates) mind explaining why on 
>its own it's better than CONTINUATION, beyond syntactic sugar?

For HEADERS it will tell the server up front how long the headers
are going to be (in compressed form) and this will aid memory
management and DoS resistance.

For DATA it will improve transmission efficiency, both in terms
of bandwidth and overhead for large objects.

As seen by the pull request offered as part of another proposal,
the entire draft becomes significantly shorter and simpler to
implement once all the special-casing of CONTINUATION disappears.

A side effect of getting a SETTING for largest frames accepted will
aid also sites that want to restict HEADER or frame sizes and provide
early warning of things that are simply not going to work because
of restrictions at the peer.  Today there is no way for HAproxy or
Varnish to say "Don't even try larger than X bytes".

---

From: "Walter H." <[email-masked]>
Date: Thu, 3 Jul 2014 10:00:26 +0200

> Uhm, we're talking about HTTP/2 frames, not ethernet frames.

that doesn't prevent you of making a mistake by design;
because a HTTP/2 frame can be of ANY size and so it makes no sense of
handling "jumbo http/2 frames";
its the lower layer that handles jumbo frames and NOT HTTP itself;

---

From: Cory Benfield <[email-masked]>
Date: Thu, 3 Jul 2014 09:19:03 +0100

> its the lower layer that handles jumbo frames and NOT HTTP itself;

You need to go back and reread the spec or any of the many, many
CONTINUATION discussions we've had. The maximum HTTP/2 frame size is
16kB - 1. This is the limitation PHK and others are arguing for
lifting with the jumbo frames proposal.

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 04 Jul 2014 00:41:51 +1200

On 2014-07-03 19:11, Greg Wilkins wrote:
> On 3 July 2014 16:36, Mark Nottingham wrote:
> 
>> 
>> On 3 Jul 2014, at 4:29 pm, Amos Jeffries wrote:
>> 
>> its

For Mark: this list by Greg pretty much sums up my reasons. I have added 
some inline comments after some entries where we differ.

>    that their is a 6th frame and you have to throw away all that work 

This is the key reason for me supporting jumbo frames. One of the key 
benefits we/Squid would get out of HTTP/2 is to shrink the buffer 
default size down closer to 16-32KB, enough for a frame or two.
  Each time CONTINUATION occurs we would have to re-allocate the entire 
buffer and move not just the already received bytes, but after reading 
the CONTINUATION we would have to shuffle the payload bytes forward over 
the CONTINUATION frame header in order to pass the encoded bytes to 
HPACK.
  Jumbo frames gives us the size to reallocate the buffer to before 
reading the entire large header set into it. Cutting away N 
reallocates-and-copy cycles per request.

>    frame to send very large content.

Willy T's numbers about Gbps line rate issues already covered the 
reasons this is important for high speed users. I can point at CERN here 
pumping science data over HTTP in 1TB sized chunks. Boinc applications 
are another set of HTTP based science transfers, but only pumping out 
vast numbers of GB chunks. Cutting the line speed by 96% is not going to 
make them happy with HTTP/2. Allowing their rare cases to negotiate 
larger DATA size ==> speed and more conversions.

>    in this part of the code will be really really really valuable.

The way this is phrased seems a bit dramatic but in my experience and 
those of some of our users, this is probably an understatement.
  The teardown issue can be seen today in HTTP/1.1 with multiple 
persistent connections pumping at more than 20K requests per second. If 
one or more clients starts causing regular connection teardown the 
system can easily run out of available sockets and effectively DoS new 
clients, and consumes noticeably more CPU. HTTP/2 is in a prime position 
to resolve this by only requiring stream teardown, but CONTINUATION 
opens the door for broken/malicious clients to emit a single frame in 
the wrong position and there goes the whole connection (again, and 
again, ...).

>    - A finally - CONTINUATIONS are jumbo frames.     If it quacks like 

... smells like a duck ...

Amos

---

From: Michael Sweet <[email-masked]>
Date: Thu, 03 Jul 2014 09:45:10 -0400

Based on some of the other discussions about max header sizes for proxies, and given the confusion that would ensue concerning "sent" and "received" header size settings, I would say there should just be one limit for both directions.

>> I was thinking of proposing an option 4, where a setting is added that specifies a total header frame(s) length limit, and a default limit of 16KB. This eliminates the HOL problems by default, and…

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 4 Jul 2014 00:08:51 +1000

​On Jul 2, 2014, at 9:20 PM, Mike Bishop <[email-masked]>
wrote:
>
> The real question with such an extension is the reverse direction -- does
the client also advertise their maximum response header size?  What should
a server do if the request a client sent produces a response the client
isn't going to accept due to header size?​

I think the options are either: downgrade to 1.1, send a 4xx response, or
send a 5xx response. I want to say 406, but not quite. In any case, with a
400 or 500 you can explain in plain text (or HTML) what the problem is, and
maybe even how to fix it (or at least to give up); not so easy with a
RST_STREAM.

On 3 July 2014 23:45, Michael Sweet <[email-masked]> wrote:

> Based on some of the other discussions about max header sizes for proxies,
> and given the confusion that would ensue concerning "sent" and "received"
> header size settings, I would say there should just be one limit for both
> directions.
>
>
Settings simply say "this is what I will accept." You don't have to
advertise what you'll send -- just send it. If you say you're willing to
receive 64K and the other guy is willing to receive 16K​, who knows, *e
might send you 64K. Odds are against it, though. Whether you're upstream or
downstream or it's a request or a response isn't relevant, at this level.
​

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Michael Sweet <[email-masked]>
Date: Thu, 03 Jul 2014 10:29:01 -0400

Right, forgot that the SETTINGS ACK doesn't contain any settings...

On Jul 3, 2014, at 10:08 AM, Matthew Kerwin <[email-masked]> wrote:

> 
> ​On Jul 2, 2014, at 9:20 PM, Mike Bishop <[email-masked]> wrote:
> >
> > The real question with such an extension is the reverse direction -- does the client also advertise their maximum response header size?  What should a server do if the request a client sent produces a response the client isn't going to accept due to header size?​
> 
> I think the options are either: downgrade to 1.1, send a 4xx response, or send a 5xx response. I want to say 406, but not quite. In any case, with a 400 or 500 you can explain in plain text (or HTML) what the problem is, and maybe even how to fix it (or at least to give up); not so easy with a RST_STREAM.
> 
> 
> Settings simply say "this is what I will accept." You don't have to advertise what you'll send -- just send it. If you say you're willing to receive 64K and the other guy is willing to receive 16K​, who knows, *e might send you 64K. Odds are against it, though. Whether you're upstream or downstream or it's a request or a response isn't relevant, at this level.
> ​
> 
> -- 
>   Matthew Kerwin

---

From: Jason Greene <[email-masked]>
Date: Thu, 3 Jul 2014 14:12:52 -0500

> Yes, this means waiting one RTT for large headers -- that's probably okay, since if our primary use-case is authentication, you need to wait one RTT to get challenged for auth to begin with.  (If y…

Ah ok you answered it. This was “this proposal”. 

> 
> The real question with such an extension is the reverse direction -- does the client also advertise their maximum response header size?  

Yes my thinking is this would be equivalent to table size, where both peers send a value (only if it differs from 16KB) in a SETTINGS frame.

> What should a server do if the request a client sent produces a response the client isn't going to accept due to header size?

That’s a good question. If we assume the input value should pass up to the request/response processing layer then I would say a 4xx error, since its the client imposing the restriction.

On the other hand if its the framing layer that enforces this, then an RST_STREAM with a new error code might be more appropriate.

---

From: Adrian Cole <[email-masked]>
Date: Thu, 3 Jul 2014 14:24:40 -0700

Proposal 4 (remove continuation; add setting for total header frame(s)
length limit) works for me. I know details are pending, but I'm
on-board with the idea.

-A

---

From: Martin Thomson <[email-masked]>
Date: Thu, 3 Jul 2014 14:34:51 -0700

> length limit) works for me. I know details are pending, but I'm

To be clear, I believe that proposal 4 is two things:

1. A setting that describes the maximum permissible compressed size of
a header block (default 16K)

2. A mandate to fill all frames carrying header block fragments if
they are followed by a CONTINUATION

In the normal case, this would mean that an implementation could avoid
even implementing CONTINUATION.

That is, if we get this right and say that padding can be used to fill
the frame, otherwise we're back in the poop.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 3 Jul 2014 14:43:57 -0700

> the frame, otherwise we're back in the poop.

Thinking on this more, I think that we need to say that the setting
can't be set to <16383, or we end up with pathological cases.

---

From: Jason Greene <[email-masked]>
Date: Thu, 3 Jul 2014 16:56:30 -0500

> can't be set to <16383, or we end up with pathological cases.

Can you share some examples you were thinking of?  Is it something like someone sets a 1 byte limit?

---

From: <[email-masked]>
Date: Thu, 3 Jul 2014 22:18:03 +0000

>the frame, otherwise we're back in the poop.

Could somebody please write a proposal for option #4? I can't even follow what the option really is.  For example, Martin is talking about CONTINUATION still, but in the quote from Adrian, the first thing he says is "remove continuation".

---

From: Nicholas Hurley <[email-masked]>
Date: Thu, 3 Jul 2014 15:22:05 -0700

I could get behind the idea of mandating a full HEADERS frame before
sending a CONTINUATION. However, I don't want the setting. People are going
to set whatever they want for their max compressed header block size (note:
NOT frame size, I still want continuation if I'm talking to an endpoint
that allows > 16k). They're going to set a limit even without this change
to the spec. If there is any header block that exceeds that limit, the
connection will most likely be failed anyway. No sense in negotiating a
setting when you can just send a GOAWAY in the exceedingly rare
pathological case.

Peace,
  -Nick

---

From: Martin Thomson <[email-masked]>
Date: Thu, 3 Jul 2014 15:30:14 -0700

> Could somebody please write a proposal for option #4? I can't even follow what the option really is.  For example, Martin is talking about CONTINUATION still, but in the quote from Adrian, the firs…

Is the above not sufficiently precise for you?  I'm guessing, but I
think that Adrian was just confused about what Jason was proposing.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 4 Jul 2014 08:58:26 +1000

Hi Greg,

Thanks (and thanks also to Amos for elaborating).

Some responses below…

On 3 Jul 2014, at 5:11 pm, Greg Wilkins <[email-masked]> wrote:
> 
> 	• The max acceptable size will be declared in a SETTING, allowing for it to be know in advance

True, but that’s the case for *individual* frames already (see below for the CONTINUATION case)

> 	• max frame size can be adjusted down for low resource impls.

OK. This seems like a general argument for JUMBO, not one regarding CONTINUATION. There are other ways to get there, and the consensus has been that a low-resource ALPN token would be the way to go here, because there are many different aspects that would need to be “tuned”, not just frame size.

> 	• A true jumbo frame has its total size in the first frame, so an impl can immediately know if it is acceptable or not. With continuations, you might have to process 5 frames to reach a 64KB limit, only then to find out that their is a 6th frame and you have to throw away all that work done.

As Amos says, this is the big one.

I’ve heard it argued that CONTINUATION on smaller-than-16k chunks is useful, because it allows you to flush your buffers on sending. However, in the current design, it seems like that’s pushing a problem off of the sender and onto the recipient, which isn’t a great dynamic. 

I’d like to hear what people think about this. It seems like, because of the nature of HTTP headers, either the sender or the recipient needs to be able to buffer them in entirety (yes, there are tricks one can play in certain situations, but that doesn’t hold in the general case).

Would it be interesting if using CONTINUATION required prefixing their size so that recipients can make this up-front decision?

Note that there are a few different forms this could take; e.g., it could be a byte count, or it could be a “reservation” of the number of continuations that the sender thinks they’re going to need.

> 	• The code path for handling large or small frames will be pretty much the same for jumbo frames, with the exception of the code that determines the frame length.   With continuations, there is a significantly different code path that will be executed only for headers that are larger than 20-something KB.   This is a code base that will be rarely used and thus probably not well tested.  Here be dragons!

Understood.

> 	• Jumbo frames can also be applied to data frames (IF both the endpoints so desire).   This can be done to either simply tune the frame size to the most efficient size for the network (eg the fast hop between a load balancer and application server maybe able to have a much larger frame size and still have good multiplexing), or it may even be used to allow a single frame to send very large content.

This drops us back into the jumbo argument. Allowing large DATA frames defeats the goals of multiplexing, and would only encourage clients to use multiple connections, putting us into the same situation with have with /1. While it’s easy to imagine non-browser, specialist uses of HTTP where both ends know that only one request will be outstanding at a time and therefore able to agree to this, in the general case this is not so. 

This is true regardless of the type of Web site, because browsers are the clients and browsers can’t make that assumption; the user might open up another tab to that site, the JS might ask for another fetch, the user might navigate, etc. etc.

That’s what led me to say that we didn’t have consensus to add jumbo frames; it’s been discussed at length, and I’m not inclined to revisit it unless there’s genuinely new information that’s capable of changing peoples’ minds. I asked this question to find out why jumbo advocates were so down on CONTINUATION, not to reconsider jumbo.

> 	• Closing stuff down is just really hard.  No its harder than that!  I mean really really difficult!   SSL handshakes, half closed TCP connections, race conditions, blah blah blah - it is just really really really difficult to write robust code that always closes neatly, without double/triple work and does not leak any resources.    Any simplification in this part of the code will be really really really valuable.

Understood, and I think widely and violently agreed to. This is a secondary effect, but well worth noting.

> 	• Jumbo frames means that the END_STREAM flag is set on the last frame of a stream.   It is a real WTF moment for anybody looking at the spec when they see CONTINUATIONS can be sent after END_STREAM.  The confusion this causes should not be underestimated and reading this list is a good indication of the communication problems that will result.

Right. That’s something in the design that’s worth exploring, if at least so that everyone understands the tradeoffs. 

> 	• A finally - CONTINUATIONS are jumbo frames.     If it quacks like a duck…. 

When you have a hammer… ;)

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 4 Jul 2014 09:49:26 +1000

> the frame, otherwise we're back in the poop.
Do you mean "padding can't be used"? Otherwise I can still send an empty
HEADERS, padded out to the full frame size, and put all my real bytes in a
CONTINUATION. Because I'm a jerk.

Conversely is this proposal oppositional to padding? If you want to obscure
the length of your header block, and you can fit within a single HEADERS
frame, then all well and good; but if you know/suspect you're going to
overflow a single frame, you're kinda SOL. If you're going to overflow by a
calculable amount, according to the current draft you can defer more bytes
to the continuation and pad out the first HEADERS. If you can only
continuate when the HEADERS is legitimately full, you can't do anything to
obscure the length. Would this proposal reopen the "padding on
CONTINUATION" issue?

And if it's not considered significant, why is the padding on headers to
start with?

---

From: Martin Thomson <[email-masked]>
Date: Thu, 3 Jul 2014 16:55:48 -0700

> CONTINUATION. Because I'm a jerk.

I'll try to remember that.

You can only add 256 bytes of padding though.

The plan here was to count the padding toward the limit.  If you feel
the need to pad, then that would not be possible if you were too close
to the cap your peer sets.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 4 Jul 2014 11:52:18 +1000

On 4 July 2014 08:58, Mark Nottingham <[email-masked]> wrote:

> That’s what led me to say that we didn’t have consensus to add jumbo
> frames; it’s been discussed at length, and I’m not inclined to revisit it
> unless there’s genuinely new information that’s capable of changing
> peoples’ minds. I asked this question to find out why jumbo advocates were
> so down on CONTINUATION, not to reconsider jumbo.

Well it is hard to express the criticism of continuations without
mentioning other solutions that meet the same requirements without the
uglyness.

If we are genuinely considering ways to remove the uglyness of
continuations, then I think it is not reasonable to reject any solution
that walks/talks/quacks like jumbo frames (as after all - continuations ARE
jumbo frames).     I think it is better to try to understand why jumbo
frames were rejected last time and try to re-propose them so that they
satisfy all the issue and meet all the objects to at least "can live with"
level.

That's exactly what a few of us are currently doing.... stand by and please
don't reject just because it walks/talks/quacks like jumbo frames.

cheers

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 4 Jul 2014 12:06:05 +1000

On 4 Jul 2014, at 11:52 am, Greg Wilkins <[email-masked]> wrote:

>  I think it is better to try to understand why jumbo frames were rejected last time and try to re-propose them so that they satisfy all the issue and meet all the objects to at least "can live with" level.

That’s what I was trying to do, Greg.

> That's exactly what a few of us are currently doing.... stand by and please don't reject just because it walks/talks/quacks like jumbo frames.

… and I didn’t; I explained why jumbo frames weren’t adopted before, and suggested another path that attempts to work around the problem that you explained. I still haven’t seen any reaction to that proposal; happy for it to be shot down, less happy to continue the meta-discussion. 

Cheers,

---

From: Michael Sweet <[email-masked]>
Date: Thu, 03 Jul 2014 23:06:44 -0400

Mark,

ALPN tokens don't address clear text HTTP/2, which I'm guessing matters to low-resource end points.

> On Jul 3, 2014, at 6:58 PM, Mark Nottingham <[email-masked]> wrote:
> 
> Hi Greg,
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 4 Jul 2014 13:10:16 +1000

Michael,

As a reminder, we’re using ALPN tokens to identify the protocol in use, regardless of whether TLS is actually in use. 

E.g., in the upgrade exchange, we use an ALPN protocol identifier; likewise in alternative-services.

See:
  http://http2.github.io/http2-spec/#versioning

---

From: David Krauss <[email-masked]>
Date: Fri, 4 Jul 2014 12:58:24 +0800

> This drops us back into the jumbo argument. Allowing large DATA frames defeats the goals of multiplexing, and would only encourage clients to use multiple connections, putting us into the same situ…

This logic sounds faulty. Reduced data frame size does not prevent an implementation from giving itself needless HOL blocking, and jumbo frames do not cause an implementation to do so.

An implementation is responsible for deciding how much data it can commit to sending before switching gears. That should be informed by the TCP window sizes.

Of course it’s possible to get tripped up, but the possibilities leading to stalls or deadlock aren’t eliminated by a fixed frame limit. They’re only even mitigated down to a given amount of data in flight. To maximize the mitigation, just let receivers declare a max frame size by SETTINGS.

Will a max frame size just re-open the need for CONTINUATION? I don’t think so. We can see that the header size distribution falls off faster than exponentially, but optimal frame size will continue to increase with network evolution. The few applications that absolutely must have 64K+ of headers can work around or deal with potentially receiving 64K or 100K frames. And again, a responsible sender shouldn’t unconditionally max out the frame size anyway.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 04 Jul 2014 07:57:06 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>I've heard it argued that CONTINUATION on smaller-than-16k chunks is 
>useful, because it allows you to flush your buffers on sending. However, 
>in the current design, it seems like that's pushing a problem off of 
>the sender and onto the recipient, which isn't a great dynamic. 

In the HTTP ecosystem the choking point(s) are everywhere but on
the client side:  proxies, load-balancers and servers all cope with
far higher traffic-densities than clients do.

There may be exceptions such as CERNS LHC TB/s data distribution,
but I bet they will not have trouble buffering a few kilobytes of
headers before sending them :-)

I think it is perfectly fair to demand that clients be able to
*send* the HEADERS in one go, because there are plenty of avenues
for them to do so, even in very small-memory environments.

Very puny environments may be able to predict or even hardcode
the HEADERS length -- after all there are only so many different
things the microcontroller in a light-bulb can tell the world.

If that doesn't work for you, you can structure you web-application
to put the variable stuff into the DATA frames.

Finally, by suitably choosing how to use HPACK compression, you
can do a two-pass algorithm and just calculate the HEADERS length
on the first pass, and send the HEADERS on the second pass, using
as many writes as you like.  If you are that memory constrained
there will be no other competing frames sent at the same time.

>This drops us back into the jumbo argument. Allowing large DATA frames 
>defeats the goals of multiplexing, [...]

When I tried to learn to play the violin, the first thing my teacher
said was "It's unfair:  You have to pay for the entire bow, but you
only get to use a little bit of it."

Just because you *can* send jumboframes doesn't mean you *have* to
do so.

The one exception I want is that you SHALL send all the headers in one
single frame.

The size of those headers is a web-app design issue, and if sending
around 40K of cookies hurts example.com's performance and
user-perception, then example.com's web-designers are perfectly
aware both what and how to fix that problem.

---

From: Mike Bishop <[email-masked]>
Date: Fri, 4 Jul 2014 16:33:05 +0000

Our preference lies somewhere between #0 and #4.  We feel like the drawbacks of CONTINUATION, while they certainly exist, are just as present in any of the other proposals.  We haven’t seen a compelling reason that another proposal is substantially better in a way that justifies an invasive change.  CONTINUATION is effectively a jumbo frame scoped only to headers -- as they should be, since there is implementation data suggesting that jumbo data frames hurt in the normal case.

It shouldn’t require an extension to relay a valid HTTP/1.1 request over HTTP/2.  That makes either solution as an extension a non-starter for us.  That said, there are smaller changes that could possibly improve the situation.

HTTP semantics already have the concept of rejecting a request because the request headers are too large; making this more transparent to the client by optionally pre-announcing these limits and allowing intermediaries closer to the client to fail the request sooner is an improvement that doesn’t change semantics.  However, the concept of limiting response header size is new, and it’s unclear how an intermediary would reasonably handle this -- “Your request was fully processed, possibly changing state, but I won’t tell you what the outcome was because it’s bigger than you can receive. RST_STREAM YOU_CANT_HANDLE_THE_TRUTH.”  What does that turn into on a /1.1 leg?  A client could not successfully retry the request in that situation.

Requiring that CONTINUATION only occur when the previous frame is max-size as a way to avoid abuse is certainly reasonable.  If that makes people more comfortable, let’s take it.  (That does make this code path even less frequently hit, though, and it’s therefore harder for Will to trigger it “just because” in his generous effort to assist with the testing of every HTTP product on the web.)

The one salient difference that I see between jumbo frames and CONTINUATION is that the recipient knows up front how big the blob is.  This could be addressed by including in a HEADERS/PUSH_PROMISE frame either the total size of the headers or the number of CONTINUATION frames which follow.  (This might be present only if END_HEADERS is not set, or could replace END_HEADERS.)

I frankly don’t care where the END_STREAM flag lives.  Flip a coin and let me know.

Sent from Windows Mail

From: Mark Nottingham<mailto:[email-masked]>
Sent: ‎Wednesday‎, ‎July‎ ‎2‎, ‎2014 ‎4‎:‎12‎ ‎AM
To: HTTP Working Group<mailto:[email-masked]>
Cc: Tatsuhiro Tsujikawa<mailto:[email-masked]>, Mike Bishop<mailto:[email-masked]>, Gábor Molnár<mailto:[email-masked]>, Patrick McManus<mailto:[email-masked]>, Shigeki Ohtsu<mailto:[email-masked]>, Stephen Ludin<mailto:[email-masked]>, Johnny Graettinger<mailto:[email-masked]>, Hasan Khalil<mailto:[email-masked]>, Jeff Pinner<mailto:[email-masked]>, [email-masked]<mailto:[email-masked]>, Adrian Cole<mailto:[email-masked]>, [email-masked]<mailto:[email-masked]>, Cory Benfield<mailto:[email-masked]>, Daniel Stenberg<mailto:[email-masked]>, Flack, Martin<mailto:[email-masked]>, [email-masked]<mailto:[email-masked]>, Greg Wilkins<mailto:[email-masked]>, Jeroen de Borst<mailto:[email-masked]>

0) the status quo

Thanks,

---

From: Jason Greene <[email-masked]>
Date: Fri, 4 Jul 2014 13:09:49 -0500

Just to be clear, option 4  is:

1) Senders must not send continuations unless they have filled and sent a maximum sized header frame.

2) Senders can not send more than MAX_HEADER_BLOCK_SIZE (or alternatively MAX_HEADER_SIZE based on decoded vs encoded size)

3) MAX_HEADER_[BLOCK_]SIZE defaults to 16KB

4) A larger MAX_HEADER_[BLOCK_]SIZE value can be specified by the sender in a SETTINGS frame 

The end result is that a proxy or a server does not have to process and/or relay unlimited data when it will ultimately reject a value <= 16KB. In the .02% cases where a value greater than that is needed, a reasonable limit is negotiated thereby mitigating the impact of HOL blocking and reducing or eliminating connection drops.

To be clear, suggestions that servers simply GOAWAY and drop the connection is not presenting an adequate solution:

1) The spec explicitly requires that all CONTINUATIONS be processed even if they are discarded. The only exception is DOS prevention. If an implementation just refused to do it, they are not really complying with the spec, and potentially excluded from walled gardens.

2) This approach effectively prevents intermediaries from coalescing connections.

There was a separate fifth proposal which was to leave continuations as is, but make them an optional extension. This 5th option is effectively a partial solution to what 4 does, only addressing the issue in the <=16KB case . Although it does have the added benefit of removing lots of complexity from the base specification, and leaves open the door for better solutions to sending > 16KB headers.

Jumbo frames was specifically excluded from this discussion, but is effectively a sixth option.

I realize not everyone cares about intermediaries (or intermediary software vendors), but they are important to the HTTP ecosystem (load balancers anyone?), and the protocol should at least make a minimal effort to address problems in this area. There are 6 workable proposals, and I am sure there are more options. I would be interested in any other solutions/ideas from those that still favor the status quo.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Sat, 05 Jul 2014 14:34:55 +0900

> 1) Senders must not send continuations unless they have filled and sent a maximum sized header frame.

This is just a minor tweak, but it should be "Senders must not send 
continuations unless they have filled the previous (header or 
continuation) frame to the maximum size.

Regards,   Martin.

---

From: <[email-masked]>
Date: Sat, 5 Jul 2014 07:44:48 +0000

>  http://http2.github.io/http2-spec/#versioning

That doesn't help for direct h2c communications (ie without upgrade).
I know it seems far-fetched right now, but it's conceivable that http/2 might actually replace http/1.x, in which case upgrade would no longer be needed and there would be no way to pre-negotiate the settings for a low-resource alpn token.

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 6 Jul 2014 10:58:20 +1000

> I know it seems far-fetched right now, but it's conceivable that http/2 might actually replace http/1.x, in which case upgrade would no longer be needed and there would be no way to pre-negotiate t…

If you’re doing a prior knowledge connection <http://http2.github.io/http2-spec/#known-http>, you still need to pre-arrange what protocol you’re going to speak; that gives you an opportunity to figure out which ALPN token to use.

---

From: Adrian Cole <[email-masked]>
Date: Sun, 6 Jul 2014 10:16:11 -0700

Thanks for clarifying this, Jason. I misunderstood earlier. Option 4
seems to really be more about guardrails.  IOTW expose a setting that
will inform the peer about header blocks > X vs just send a go away
and have them guess why.

>From a client POV, and where the client only sends 1 header frame,
this seems to simplify processing. Since the max the client can
receive is one header frame, it only needs to deal with the error case
when the server couldn't fit its response into that.

If on the other hand the client deviates from default, or pays
attention to dynamic settings from the peer, we have more moving
state, and more work than status quo. This is opt-in complexity,
though.

I'm still ok with option 4.

If I've again misunderstood something, do call me out!
-A

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 7 Jul 2014 18:10:25 +1000

Just a few comments after letting this sink in...

> 1) Senders must not send continuations unless they have filled and sent a maximum sized header frame.

Is there any motivation for #1 beyond accommodating implementations that don't want to think about CONTINUATION?

> 2) Senders can not send more than MAX_HEADER_BLOCK_SIZE (or alternatively MAX_HEADER_SIZE based on decoded vs encoded size)

This needs to include padding if the intent is to control use of CONTINUATION.

> 3) MAX_HEADER_[BLOCK_]SIZE defaults to 16KB

Can it be set lower?

Also, I think this needs to default to 2**14 - 10 to account for HEADERS (w/ priority flag) overhead, if the intent is to default to 0 continuations.

> 4) A larger MAX_HEADER_[BLOCK_]SIZE value can be specified by the sender in a SETTINGS frame 

What do you mean by "sender" here?

> The end result is that a proxy or a server does not have to process and/or relay unlimited data when it will ultimately reject a value <= 16KB. In the .02% cases where a value greater than that is…

SETTINGS is explicitly not a negotiation mechanism; it only allows one end to advertise its configuration to its peer.

In this case, the semantic of MHS (MAX_HEADER_SIZE) would be roughly "I allow headers coming at me to be at most THIS BIG." There's no way for the peer to ask for more; it has to accept the limitation.

Thinking through deployment scenarios --

a) Imagine a forward proxy that has a single connection to the user agent. Each time the UA requested an origin that necessitated a new connection, the proxy would have to advertise a new, lower request MHS if the origin advertises one less than the current client connection's. It could only raise the UA->proxy request MHS again when that connection is no longer in use by the client (and it'd likely need to use some heuristic to figure this out). This isn't great, because you're effectively getting the smallest MHS you're currently connected to (for some value of "currently"), and it's dynamic.

Alternatively, it could keep the UA->proxy request MHS constant and synthesise a 431 error response when an origin's request MHS was violated. What's bad here is that something that works without a proxy won't work when the proxy is interposed (and we know how those support calls go). Arguably, though, this isn't much different from e.g. Squid's 20K header limit.

b) In the case of a "reverse" proxy / gateway that multiplexes many UA connections into one origin connection, the proxy's advertised response MHS would need to be the smallest of the UA connections', for the lifetime of that connection. Again, it'd need to be adjusted as clients came and went, but this is pretty nasty, since it gives clients control over others' sessions. 

Alternatively, the gateway could synthesise a 502 (for example) back to the UA that's advertise too low a MHS for a given response. Not sure if it'd be useful to communicate the problem back to the origin (especially since it'd have to be HTTP/2-specific)... 

Are there any other ways to handle it?

Of course, if MHS can't be set to lower than 16K (-1), there's at least a floor, and if it never gets advertised above 16K (-1) by anyone, you don't really see these issues. The question is how likely that is to be the case.

The other interesting aspect to think about is how MHS surfaces in HTTP APIs. Request MHS seems like it would necessitate a synthesised 431 whenever the client attempts to make a request that violates server policy. Response MHS is tricker; by default, the response would just get dropped on the floor, in the same way that a client that doesn't like your response doesn't give you any feedback. However, I suspect that people would invent new APIs on top of the "normal" HTTP API to allow this information to come through...

That's how I currently understand it; feel free (as always) to poke holes.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 10:54:09 +0200

Hi Mark,

making a quick comment about this below :

On Mon, Jul 07, 2014 at 06:10:25PM +1000, Mark Nottingham wrote:
> a) Imagine a forward proxy that has a single connection to the user agent.
> Each time the UA requested an origin that necessitated a new connection, the
> proxy would have to advertise a new, lower request MHS if the origin
> advertises one less than the current client connection's. It could only raise
> the UA->proxy request MHS again when that connection is no longer in use by
> the client (and it'd likely need to use some heuristic to figure this out).
> This isn't great, because you're effectively getting the smallest MHS you're
> currently connected to (for some value of "currently"), and it's dynamic.

I think you're mixing two sides. Here the proxy does not have to impact one
client's MHS depending on the other ones. At best it would impact the MHS on
the connections to the origin server. The MHS is a property of the connection
and does not have to be transferred between the two sides (better, must not).

>From this point on, the proxy can decide whether or not some client connections
may be coalesced to the origin server. If one is watching a video while another
one is loading some CSS, it's likely that we'll make better use of at least two
connections. Just like we'd prefer to have two if one is watching a video at
16kB data frames while someone else is doing some very interactive stuff like
using a WYSIWYG editor or simply benefiting from auto-completion on a search
engine.

Thus I'm pretty sure that forward proxies (and probably reverse proxies as
well) will try to assign streams to connections based on some form of QoS.
If it has to be done for data, for sure it will be needed on headers. Imagine
the large number of small companies still running on ADSL with 4M/1M or so,
you definitely don't want there to have one user sending large cookies over
the connection and block the stream for all other users for the time it takes
to send a huge 16kB of headers.

So MHS can help the proxy take the proper decision as well.

Best regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 7 Jul 2014 18:59:49 +1000

Hey Willy,

> and does not have to be transferred between the two sides (better, must not).

I'm not sure I buy that. If I connect to a proxy and its request MHS is bigger than the next-hop's request MHS, I'll send requests assuming the first. 

It's true it doesn't have to be in lockstep, but there are consequences they're if not (as I talked through).

> So MHS can help the proxy take the proper decision as well.

I suspect some intermediaries will end up grouping requests from different connections together by MHS.

I wonder if any sort of MHS setting is better-off being something coarse-grained (e.g., aligned on 1k or 4k) so that you don't have peers doing weird things like advertising 423 byte MHS, thereby screwing things up...

Cheers,

---

From: <[email-masked]>
Date: Mon, 7 Jul 2014 09:11:27 +0000

On Monday,07 July 2014 11:00, [email-masked] wrote:
> I wonder if any sort of MHS setting is better-off being something coarse-grained
> (e.g., aligned on 1k or 4k) so that you don't have peers doing weird things like
> advertising 423 byte MHS, thereby screwing things up...

I think that's a good idea. If the max is defined as 'max = MHS*K', where K=1k or 4k or whatever.  Depending on the value selected for K, that could also provide some future-proofing in case, for whatever reason, the length field were to be made bigger.

What about doing the same thing for flow control values (i.e. coarse-grained values aligned on 1k or 4k or...)?

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 11:48:08 +0200

On Mon, Jul 07, 2014 at 06:59:49PM +1000, Mark Nottingham wrote:
> > Here the proxy does not have to impact one
> I'm not sure I buy that. If I connect to a proxy and its request MHS is
> bigger than the next-hop's request MHS, I'll send requests assuming the
> first. 

I mean, you're the UA, you don't need to know anything about the server since
your connection is with the proxy. Let's not repeat the proxy-connection hell!
If the proxy accepts 64kB because you're authenticating to it using kerberos
tickets, but it needs only 16kB to go outside, what matters for the UA is the
proxy connection only. And conversely, a reverse-proxy could need a larger MHS
with the origin than what it uses with the client because it passes additional
information (eg: SSL cert). So the two are definitely independant.

> connections together by MHS.

Yes that's what I was suggesting as well.

> I wonder if any sort of MHS setting is better-off being something
> coarse-grained (e.g., aligned on 1k or 4k) so that you don't have peers doing
> weird things like advertising 423 byte MHS, thereby screwing things up...

I definitely agree. Using powers of two, or at maybe half-steps would ensure
easy allocation/grouping.

Cheers,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 09:49:52 +0000

In message <[email-masked]>, Willy Tarreau writes:

>> advertises one less than the current client connection's.

The proxy might instead choose to not pass the lower limit, but return
413's if the conflict arises.

Since client side proxies are almost always closer to the client, this
may make good sense.

Also note that the proxy holds different shared compression state
with the backend, it may be able to squeeze the request through.

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 7 Jul 2014 12:17:04 -0400

>
> “Your request was fully processed, possibly changing state, but I won’t
> tell you what the outcome was because it’s bigger than you can
> receive. RST_STREAM YOU_CANT_HANDLE_THE_TRUTH.”
>

Lol, can we add this error code? Please? /jk

Agreed. The concept of pre-announcing request and response header limits
seems... messy to me. I don't know what it means if I have a request who's
header size is larger than the limit. What do I do with that? I can't
reasonably "trim" the headers without deeper application knowledge: they
are what they are. How do I surface that error to the user, and what are
they supposed to do about it? For that matter, how does the remote endpoint
discover the impact the setting is having on actual clients? I'd rather
just optimistically fire off the request as I do now, and let the remote
endpoint be the arbiter of whether it's over limit.

It also seems problematic to announce to a potentially-malicious host
exactly how much they can get away with before DoS mitigation starts
kicking in.

> Requiring that CONTINUATION only occur when the previous frame is max-size
> as a way to avoid abuse is certainly reasonable.  If that makes people more
> comfortable, let’s take it.  (That does make this code path even less
> frequently hit, though, and it’s therefore harder for Will to trigger
> it “just because” in his generous effort to assist with the testing of
> every HTTP product on the web.)
>

This is a concern. I don't think it's at all desireable to have a part of
the spec which is designed to rarely be used. The risk of incorrect
implementation is too high.

Chromium will fully abide by whatever this working group decides, but right
now h2-13 is the implementation draft, and to ensure that CONTINUATION is
properly covered in h2-13 Chromium will emit them at smaller payload sizes.
This will be contentious for some, and I apologize for that, but the
alternative of an untested CONTINUATION frame is worse.

I frankly don’t care where the END_STREAM flag lives.  Flip a coin and let
> me know.

Agreed. If moving this flag makes CONTINUATION significantly more
palatable, I've no objections.

>   *From:* Mark Nottingham <[email-masked]>
> *Sent:* ‎Wednesday‎, ‎July‎ ‎2‎, ‎2014 ‎4‎:‎12‎ ‎AM
> *To:* HTTP Working Group <[email-masked]>
> *Cc:* Tatsuhiro Tsujikawa <[email-masked]>, Mike Bishop
> <[email-masked]>, Gábor Molnár <[email-masked]>, Patrick
> McManus <[email-masked]>, Shigeki Ohtsu <[email-masked]>, Stephen
> Ludin <[email-masked]>, Johnny Graettinger <[email-masked]>,
> Hasan Khalil <[email-masked]>, Jeff Pinner <[email-masked]>,
> [email-masked], Adrian Cole <[email-masked]>,
> [email-masked], Cory Benfield <[email-masked]>, Daniel
> Stenberg <[email-masked]>, Flack, Martin <[email-masked]>, [email-masked], Greg
> Wilkins <[email-masked]>, Jeroen de Borst <[email-masked]>
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 20:06:51 +0000

In message <CAEn92TpmfYG20dphWw3uhYQy8aA+sAiMpsm+taQM=[email-masked]>
, Johnny Graettinger writes:

>header size is larger than the limit.

It means you might as well not send it, the best you can hope for is
a 413, at worst, it will kill the connection.

Your two options is to give up, or try to spend more CPU time finding
a more efficent compression (provided the chosen header compression
allows this).

>For that matter, how does the remote endpoint

We can't prevent you from doing this, and you'll get the treatment
the protocol specifies for it.

>exactly how much they can get away with before DoS mitigation starts

This limit is unlikely to be the entire DoS mitigation mechanism or
for that matter trigger.

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 7 Jul 2014 16:43:20 -0400

> a 413, at worst, it will kill the connection.

Right. Assuming I've already done the best I can with compression, this
outcome is functionally no different than actually sending the request, and
getting a 413 or closed connection. If I send the request, at least I've
given the server visibility into the failure.

> This limit is unlikely to be the entire DoS mitigation mechanism or

Right. Servers must still handle headers which are larger than they'd like.

So, having a maximum header size setting isn't useful to the client, and it
doesn't protect the server. What's being bought here?

It *might* buy the server implementer the convenience of simply closing the
connection, rather than issuing a 413 and preserving the connection. But
then again probably not, as they'll probably want to issue 413's in some
cases anyway as part of layered DoS mitigation.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 21:16:15 +0000

In message <[email-masked]>, Johnny Graetting

>given the server visibility into the failure.

I thought about server visibility along these lines:

In the case where the server is producing the basis for subsequent
requests (ie: links on a web-page), It is reasonable to expect the
web-designer to either theoretically or through tests confirm that
the links pass muster.

In the likely case where he merely launches his browser and clicks
away, it doesn't matter if the browser or the server tells him
"413", as long as the browser shows them.

If he doesn't give this issue any thought at all and doesn't test
it either, he fails the "you must be this smart to be a web-designer
test" and we have no obligation to waste time on him, in particular
not if he got in this situation by exceeding the default 16KB HEADERS
limit.

In the other case, for instance where the client composes requests
based on some written API manual, I think the risk of huge HEADERS
is much more likely, but here again it doesn't matter who produces the
413, the right person still receives it.

So I fail to see server visibility as important.

>So, having a maximum header size setting isn't useful to the client [...]

Not in all cases, but it does for instances it tell the client that
they can get their kerberos tickets through.

And it does help in shaving DoS attaks.

A server can advertise a SETTINGS which is narrowly tuned to allow
all legitimate requests.

Any client disrespecting this with intent to DoS will declare it's
intention in the first four bytes of the HEADERS frame and the
server will instantly know, without opening the time/space window
from the valid sizes to the current drafts "unlimited" to abuse.

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 7 Jul 2014 18:21:53 -0400

> web-designer to either theoretically or through tests confirm that

This is a tall order :) Consider even a mildly-dysfunctional organization
with multiple teams owning siloed services operating over a shared cookie
space. Throw in some AJAX requests with metadata made across those service
boundaries for extra fun.

> 413, the right person still receives it.

How do I (as an API provider) discover that users are trying to use my
service in a way I'm not currently supporting?

Perhaps they're attempting a use case I'd actually like to support.
Realistically, I need to have metrics on this.

> And it does help in shaving DoS attaks.

How? Advertising the allowable length makes it easy for an attacker to
craft the most-damaging payload possible. If I tighten my limit in response
to a DoS, I also certainly don't want to advertise it, as that makes it
easy for the attacker to follow along. I therefore need a mitigation on the
server-side capable of rejecting over-limit requests, and since I must have
this anyway, what is the setting buying me?

> all legitimate requests.

In my experience, "narrow tuning to allow all legitimate requests" is an
oxymoron :)

> from the valid sizes to the current drafts "unlimited" to abuse.

If I have intent to DoS, I'll send you exactly as much payload as you allow
and no more. I'm not going to make it easy for you.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 22:27:11 +0000

In message <[email-masked]>, Johnny Graetting
er writes:
>--001a11c1bbc0fda6b704fda1e67e
>Content-Type: text/plain; charset=UTF-8
>>

I have no problems with them having to think.  Their users/customers
will probably thank me for the better performance :-)

>service in a way I'm not currently supporting?

They'll complain to you ?

>In my experience, "narrow tuning to allow all legitimate requests" is an

Well, your experience is obviously different from mine (proxied through
from the varnish users I talk with).  Their current response to the
lack of header-compression in HTTP/1.1 is to think a lot about how
big their ingoing and outgoing headers are, and how to use this to
discriminate DoS attacks.  Some of them have both upper and lower
bounds on valid requests.

>and no more. I'm not going to make it easy for you.

That's a lot easier for me to handle than the current drafts "any
number of CONTINUATION" packets with no specified timeout.

---

From: <[email-masked]>
Date: Mon, 7 Jul 2014 22:34:06 +0000

> On 08 Jul 2014, at 00:26, "[email-masked]" <[email-masked]> wrote:
>
> How do I (as an API provider) discover that users are trying to use my service in a way I'm not currently supporting?

Set HEADERS_FRAME_SIZE = 2^31-1 (the max value, which approximates unlimited headers with CONTINUATION) and send back 413.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 15:38:28 -0700

On Mon, Jul 7, 2014 at 3:27 PM, Poul-Henning Kamp <[email-masked]>
wrote:

> In message <
> [email-masked]>,
> Johnny Graetting
> er writes:

Turns out that coordination across 30k developers isn't trivial. :)

> They'll complain to you ?

Sadly, only sometimes.
-=R

> number of CONTINUATION" packets with no specified timeout.

You're conflating things.
Timeouts when receiving metadata/headers is an issue regardless of the use
large frames or continuations, as a promise of X bytes (or a complete
header) doesn't mean it will actually happen, or that it will happen in a
reasonable period of time.
This is an issue with both large frames and continuations. It is not
limited solely to continuations.

-=R

---

From: Jason Greene <[email-masked]>
Date: Mon, 7 Jul 2014 19:11:14 -0500

On Jul 7, 2014, at 5:38 PM, Roberto Peon <[email-masked]> wrote:

> Timeouts when receiving metadata/headers is an issue regardless of the use large frames or continuations, as a promise of X bytes (or a complete header) doesn't mean it will actually happen, or that it will happen in a reasonable period of time.
> This is an issue with both large frames and continuations. It is not limited solely to continuations.

The promise is significantly easier to meet though because its limited, especially by default where the limit is 16K. You can of course still have a total connection stall, but that affects all streams anyway. What the limit does is improve the quality of the multiplexing.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 17:53:28 -0700

The limit on max header size partially solves a problem (see other thread
for the problems with how a proxy picks the right max size..) that is
different from the jumbo vs continuation stuff, though.
The limit on max frame size absolutely helps to make multiplexing work (and
I can get behind that).

I see a few distinct and orthogonal topics:
1) declaring a limit on the compressed header size
2) headers get buffered at sender until complete and sent with declared
length
3) declaring a maximum accepted frame size for headers or data frames or
perhaps all frames.
4) changing the base framing format to have a larger, but consistent size
field (e.g. from 16 bits to 24).
5) changing the base framing format to have a variable length size field
6) headers transmitted using multiple frames (i.e. continuations)
7) moving the flags on continuations to the last frame, possibly renaming
those/changing the opcodes
8) nonblocking headers
9) flow control for metadata (i.e. headers).

A few of these get conflated often (e.g. 1,2,{4,5}), despite the fact that
they're separable.

I'm saying this in the hope that we can talk about these separately.
>From my perspective:
#1: Sure, though I suspect its utility will be minimal when
proxies/gateways are in the mix.
#2: Don't like it-- it restricts implementations in ways that affect their
performance/efficiency without providing clear DoS-protection/scaling
benefit.
#3: Sure, seems reasonable to do.
#4: Fine, so long as #3 covers data and headers, though I suspect that
adding to the size field will increase protocol overhead in the common
case, and that a 16 bit length field is good enough for that common case.
#5: Don't like it, never will. It makes base framing more complicated, and
the base framing should be the simplest and most regular thing in the
protocol. Interpretation can have more complexity-- framing shouldn't.
Getting the framing wrong leads to security nightmares.
#6: I prefer fragmentation to varying the base framing due to the reasons
described in #5.
#7: Sure. We could also do PARTIAL_HEADERS* HEADERS, or HEADERS
CONTINUATION*, or SYN_STREAM (HEADERS)* END, whatever. All make just as
much sense to me as the current layout.
#8: Still for it, though it does add complexity. Probably not worthwhile to
solve the problem for HTTP-for-browsing, but useful for other usecases
where HTTP would be used.
#9: Same as #8.

-=R

On Mon, Jul 7, 2014 at 5:11 PM, Jason Greene <[email-masked]>
wrote:

> > Timeouts when receiving metadata/headers is an issue regardless of the
> use large frames or continuations, as a promise of X bytes (or a complete
> The promise is significantly easier to meet though because its limited,
> especially by default where the limit is 16K. You can of course still have
> a total connection stall, but that affects all streams anyway. What the
> limit does is improve the quality of the multiplexing.
>
> --
> Jason T. Greene

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jul 2014 14:07:50 +1000

Thanks for the detail, Roberto.

On 8 Jul 2014, at 10:53 am, Roberto Peon <[email-masked]> wrote:

> The limit on max header size partially solves a problem (see other thread for the problems with how a proxy picks the right max size..) that is different from the jumbo vs continuation stuff, though.
> The limit on max frame size absolutely helps to make multiplexing work (and I can get behind that).
> 
> 2) headers get buffered at sender until complete and sent with declared length
> 3) declaring a maximum accepted frame size for headers or data frames or perhaps all frames.
> 4) changing the base framing format to have a larger, but consistent size field (e.g. from 16 bits to 24).
> 7) moving the flags on continuations to the last frame, possibly renaming those/changing the opcodes
> 8) nonblocking headers
> A few of these get conflated often (e.g. 1,2,{4,5}), despite the fact that they're separable.
> 
> #1: Sure, though I suspect its utility will be minimal when proxies/gateways are in the mix.

See previous discussion. I'm not seeing strong pushback on doing this kind of setting, so it sounds like we're going to do something like this.

This would partially mitigate #550, and substantially help #551.

> #2: Don't like it-- it restricts implementations in ways that affect their performance/efficiency without providing clear DoS-protection/scaling benefit.

I'm hearing mixed messages on this, but how it turns out depends on other decisions we make, so probably best to punt for now.

> #3: Sure, seems reasonable to do.

And I'm not hearing people pushing back on that (assuming we make frames bigger).

> #4: Fine, so long as #3 covers data and headers, though I suspect that adding to the size field will increase protocol overhead in the common case, and that a 16 bit length field is good enough for that common case.

The only outright reluctance I hear is from Nick, and Greg has just responded to that. We'll see how that plays out, but to me the important question here is "how big?"

The answer here seems very intertwined with #6. E.g., if we chose a 16-bit frame size, that would avoid any extra overhead, but we'd still need CONTINUATION if we wanted to support large header blocks. 

OTOH if we chose 16 bits and declared that any header block over 64K-10 (compressed) needs to be supported by an extension (presumably, CONTINUATION in a separate document), that would be another approach. I'd be interested to hear what people think of this. 

If neither of those options is palatable, we'll need more bits - and that has its own set of tradeoffs.

> #5: Don't like it, never will. It makes base framing more complicated, and the base framing should be the simplest and most regular thing in the protocol. Interpretation can have more complexity-- framing shouldn't. Getting the framing wrong leads to security nightmares.

I suspect many feel the same.

> #6: I prefer fragmentation to varying the base framing due to the reasons described in #5.

That's a false choice; there are other options.

> #7: Sure. We could also do PARTIAL_HEADERS* HEADERS, or HEADERS CONTINUATION*, or SYN_STREAM (HEADERS)* END, whatever. All make just as much sense to me as the current layout.

This seems to be widely held view.

> #8: Still for it, though it does add complexity. Probably not worthwhile to solve the problem for HTTP-for-browsing, but useful for other usecases where HTTP would be used.
> #9: Same as #8.

Let's defer these until we get a sense of where the rest is going.

Thanks,

---

From: Martin Thomson <[email-masked]>
Date: Mon, 7 Jul 2014 21:25:54 -0700

>> 1) declaring a limit on the compressed header size
[...]
> This would partially mitigate #550, and substantially help #551.

I haven't repeated myself here, because I don't like doing that, but I
have pointed out in the past the poor correlation between compressed
frame size and the actual state commitment required.

The conclusion I'd drawn there being that a limit on encoded header
block size doesn't really do anything to limit the state commitment
required to process a request or response.  There is definitely some
utility in having a limit, but it's marginal.

The setting really only becomes useful - and arguably necessary - when
you consider the expanded frame size.

>From that perspective, I'd rather consider the setting in combination
with the expanded frame size.  If we were to consider the addition of
just the setting, I think that I'd be sad that you had made the
protocol have more moving parts for no real gain.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 15:36:06 +1000

On 8 July 2014 14:25, Martin Thomson <[email-masked]> wrote:

> but I

but I have pointed out in the past that the encoder header size is a
reasonable indication of additional memory requirements represented by the
header block.   The highly compressed fields within a header block are the
indexed ones, and they reference memory in the header set that is already
constrained by a setting.

The setting at least tells a receiver what memory commitment is required to
analyse the header to discover it's true size.   It may be that some
implementation may accept a header based on the limit applied to the
encoded size, but then still decide to 431 it because of the expanded size.

cheers

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 8 Jul 2014 21:24:24 +0900

On Tue, Jul 8, 2014 at 1:07 PM, Mark Nottingham <[email-masked]> wrote:

> > The limit on max header size partially solves a problem (see other
> thread for the problems with how a proxy picks the right max size..) that
> is different from the jumbo vs continuation stuff, though.
> See previous discussion. I'm not seeing strong pushback on doing this kind
> of setting, so it sounds like we're going to do something like this.
>
> > #2: Don't like it-- it restricts implementations in ways that affect
> their performance/efficiency without providing clear DoS-protection/scaling
> benefit.
>
> I'm hearing mixed messages on this, but how it turns out depends on other
> decisions we make, so probably best to punt for now.
>
> The only outright reluctance I hear is from Nick, and Greg has just
> responded to that. We'll see how that plays out, but to me the important
> question here is "how big?"
>
> The answer here seems very intertwined with #6. E.g., if we chose a 16-bit
> frame size, that would avoid any extra overhead, but we'd still need
> CONTINUATION if we wanted to support large header blocks.
>
> OTOH if we chose 16 bits and declared that any header block over 64K-10
> (compressed) needs to be supported by an extension (presumably,
> CONTINUATION in a separate document), that would be another approach. I'd
> be interested to hear what people think of this.
>
>
​If we are going to expand length bits, I very much support this approach.
We get 64K - 1 payload and Kerberos headers is supported which is only
known use case I heard so far.
By requiring implementations to fully process 64K frame, we don't need
additional settings at all (and no WINDOW_UPDATE abuse).  Implementations
are free to return 431 or RST_STREAM within their limits but just keep
running HPACK decompressor to stay connected.
This is far more simpler than the proposal and not invasive at all.  We
just goes back to the state when we decided that frame payload size should
be 16K.

​Best regards,

Tatsuhiro Tsujikawa​

> > #5: Don't like it, never will. It makes base framing more complicated,
> and the base framing should be the simplest and most regular thing in the
> > #8: Still for it, though it does add complexity. Probably not worthwhile
> to solve the problem for HTTP-for-browsing, but useful for other usecases

---

From: Michael Sweet <[email-masked]>
Date: Tue, 08 Jul 2014 09:02:59 -0400

On Jul 8, 2014, at 8:24 AM, Tatsuhiro Tsujikawa <[email-masked]> wrote:
> ...
> We get 64K - 1 payload and Kerberos headers is supported which is only known use case I heard so far.
> By requiring implementations to fully process 64K frame, we don't need additional settings at all (and no WINDOW_UPDATE abuse).  Implementations are free to return 431 or RST_STREAM within their limits but just keep running HPACK decompressor to stay connected. 
> This is far more simpler than the proposal and not invasive at all.  We just goes back to the state when we decided that frame payload size should be 16K.

32-64k is probably the sweet spot for 1080p video, and 64k is what we use for printing today.  So if we deployed HTTP/2 right now, using 16 bits for the frame length would address those two common use cases.

That said, the downside of stopping at 64k-1 is that it will force an extension or new version of HTTP if it turns out that 64k-1 isn't enough for emerging content like 4k streaming video.  On the few 4k video files I have, the average video/audio frame size varies from 17k for a highly-compressed Blender animation to 101k for an underwater video sequence.  I'll grant that most video streaming applications do employ buffering to minimize the impact of interruptions in the data stream, but we should also try to design a protocol that can scale moderately well and allow clients and servers to deliver content in pieces large enough to do useful work (see my other post on the subject).

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 08 Jul 2014 15:33:51 +0000

In message <[email-masked]>, Roberto Peon wri
tes:

Thanks for giving us a taxonomy of issues Roberto, even though
some of them tangles, it does add structure.

>1) declaring a limit on the compressed header size

I would s/a/the/ because without such a limit there is no DoS
resistance, and all implementations I know have such a limit, one
way or the other.

The question is if we tell peers only when they execeed it or also
as a interop-parameter up front.

I think there is a value in declaring it, it would make some things
easier, for instance server side proxies can autoconfigure to their
backends.

The argument that it allows DoS attacks to autotune for maximum effect
is not convincing to me, since the main opening in the current draft
is that you don't know *when* the headers will be done, more than what
size they'll end up having.

>2) headers get buffered at sender until complete and sent with declared length

This is crucial for implementation efficiency at receivers, be it
proxies or servers.  Knowing what comes makes memory allocation
much more efficient and it also informs a DoS policy with just about
the most crucial bit of data.

It could also allow opportunistic load-balancers to make "have session
cookie/does not have session cookie" decisions entirely on the length
field.

Implicit here is that by pushing this responsibility on the client,
the server is free to assume that once we have the length, the rest
will follow shortly, and DoS-mitigating timeouts can be on the order
of RTTs.

I have still not seen a real-world or strawman description
where "streaming headers" would be useful.

Yet you consistently defend this functionality as important
to you.

Please tell us what or how you would use it so we can gauge the
relevancy of this feature.

>3) declaring a maximum accepted frame size for headers or data frames or perhaps all frames.

The requirement to do so follows directly from abandonning 16 KB as the
hard and universal upper size.

>4) changing the base framing format to have a larger, but consistent size field (e.g. from 16 bits to 24).

NB: in draft -13 the field is only 14 bits

16 bits would certainly allow a much larger fraction of present day
HTTP requests to fit in a single DATA frame, but given that display
quality continually increases, that will erode rather rapidly as 4K
displays penetrate the market.

I'm not sure if it has been broken, but the last single-TCP land-speed-record
I remember was 2.5 Gbit/sec, which gives:

	At 14b length = 19000 frames/sec
	At 16b length = 2800 frames/sec
	At 24b length = 19 frames/sec
	At 31b length = 0.15 frames/sec

I think we should go with R1+31b for futures sake.

I can live with R8+24b as a more conservative starting point,
provided the 8 reserved bits are not poached for other purposes
right away.

>5) changing the base framing format to have a variable length size field

I *really* don't want this, unless there are no other options.  The
implementations which serious needs for speeds do not need to have
a complex cache-busting operation at this point in their processing.

>6) headers transmitted using multiple frames (i.e. continuations)

This one depends a LOT on the semantics of the compression, and the
outher limits proscribing possible behaviour.

If we talk about sending multiple smaller frames freight-train
style, ie: adjacent in bytes *and* in time, I could live with it.

But all we have done in that case is to have multipart jumbo frames,
with higher processing overheads and more failure modes, and their
only conceiveable benefit is that an implementation can used a
smaller fixed size transmit or receive buffer -- which is really
pointless, given that TCP is byte-serial, not frame-serial.

If we accept 2), 6) becomes pointless almost by definition.

If we are talking about anything else, including the current draft
-13, it becomes both a DoS concern, a HOL-blocker prone mechanism
and it introduces a host of corner cases, failure modes and needs for
timeouts because of the shared compression state lock-out effect.

There are many ways the current draft -13 can be improved, specifying
limits, timeouts, dropping compression shared state and so on.

But all of these workarounds are far more complex and prone to
interop trouble, than allowing larger frames and putting the entire
header-set in one frame.

>those/changing the opcodes

	Begin at the beginning
	and go on until you come to the end.
	Then stop.
		-- Alice in wonderland

If we did so, there is no need for the CONTINUATION frame type,
you can just send HEADERS or PUSH_PROMISE until you're done (as indicated
by flag)

This would make both the specification and implementations simpler,
but it would not resolve the DoS opportunity of headers being
unconstrained in time and space, nor the multiplex restrictions
imposed by the shared compression state.

>8) nonblocking headers

I'm not sure I understand what you are refering to here.  Is this
where only the first frame of the header-set is compressed using
shared state and the rest not ?

If so:  I don't like it.  There are simpler and saner solutions.

>9) flow control for metadata (i.e. headers).

I though that the connection window was intended to allow balancing
multiple different http/2 connections against each other on a
limited bandwidth connection.  A client concern mostly.

I would make all frames subject to the connection window, that way
every single extension to HTTP/2 does not have to address this
issue from the start, it will be solved once and for all.

(A related question:  If I have a window of 8 bytes, am I allowed
to send the frame header, or do I need to wait until the window is
big enough to contain the entire frame before I can send any of the
frame ?  I pressume it must be the former, since the latter could
lead to dead-lock ?)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 08 Jul 2014 15:41:48 +0000

In message <CABkgnnVevm3ar2EG_S8iQ-7sXEPcsTFYAao=GP=D83zS=[email-masked]>, Martin Thomson w
>utility in having a limit, but it's marginal.

The limit is not on what amount of state will be needed later on,
but on how much state will be needed to get to the point where the
receiver can judge the (non-)hostility of the request and estimate
how much state will be needed later on.

>you consider the expanded frame size.

It would be useful also for the unlimited in time&space CONTINIUATION
scheme in draft -13, but would need to specify not only the aggregate
size, but also the count of frames and the maxium duration of their
reception.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 9 Jul 2014 00:50:09 +0900

On Tue, Jul 8, 2014 at 10:02 PM, Michael Sweet <[email-masked]> wrote:

> 32-64k is probably the sweet spot for 1080p video, and 64k is what we use
> for printing today.  So if we deployed HTTP/2 right now, using 16 bits for
> the frame length would address those two common use cases.
>
> That said, the downside of stopping at 64k-1 is that it will force an
> extension or new version of HTTP if it turns out that 64k-1 isn't enough
> for emerging content like 4k streaming video.  On the few 4k video files I
> have, the average video/audio frame size varies from 17k for a
> highly-compressed Blender animation to 101k for an underwater video
> sequence.  I'll grant that most video streaming applications do employ
> buffering to minimize the impact of interruptions in the data stream, but
> we should also try to design a protocol that can scale moderately well and
> allow clients and servers to deliver content in pieces large enough to do
> useful work (see my other post on the subject).
>
>
Why do we have to use same frame size for HTTP/2 with video's?
We can deliver multiple DATA frames for 1 video frame.  Also big frame
really hurts multiplexing many people stated earlier.

Best regards,
Tatsuhiro Tsujikawa

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 08 Jul 2014 16:03:01 +0000

In message <CAPyZ6=KeAcET3ocnX3EDFoL38G=[email-masked]>, Tatsuhiro Tsujik
awa writes:

>We can deliver multiple DATA frames for 1 video frame.

See for instance earlier messages about the overhead cost of small frames
relative to printers.  A lot of embedded equipment processes high bandwidth
with little CPU power these days.

>Also big frame really hurts multiplexing many people stated earlier.

It is not the size of the frame that hurts multiplexing, it is the
amount of time it takes to move it, also known as "bandwidth".

Designing the protocol for only the lowest bandwidths is not a goal.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 9 Jul 2014 01:20:07 +0900

On Wed, Jul 9, 2014 at 1:03 AM, Poul-Henning Kamp <[email-masked]>
wrote:

> In message <CAPyZ6=KeAcET3ocnX3EDFoL38G=
> [email-masked]>, Tatsuhiro Tsujik
> awa writes:
>
​Multiple DATA frames can consume large bandwidth too and thereby deliver
large data quickly.

So the goal of HTTP/2 is to design it for low power embedded devices?

​Best regards,
Tatsuhiro Tsujikawa​

---

From: Michael Sweet <[email-masked]>
Date: Tue, 08 Jul 2014 12:21:13 -0400

> Why do we have to use same frame size for HTTP/2 with video's?

I'm not sure I understand the question.

> We can deliver multiple DATA frames for 1 video frame.  Also big frame really hurts multiplexing many people stated earlier.

The question is delivering those multiple frames to ensure smooth playback of the video.  In this case, using a slightly larger frame will ensure that each frame contains a full video frame (or the rest that started in the previous DATA frame).

This is a balancing act with priority, which can also hurt multiplexing.  At least with (slightly) larger frame sizes you can "atomically" transfer a full video frame and its associated audio, making it unnecessary to crank up the priority just to ensure you get the data when you need it - smooth and steady is what you want for video, not high volume bursts.

Let's also not forget that the proposal on the table is NOT to allow an endpoint to unilaterally decide to send a 1GB DATA frame, to the detriment of all.  Instead, it allows the receiver to tell the sender what its limit is (based on resources, available bandwidth, or any other criteria), which then allows the sender to adapt accordingly.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 09:31:39 -0700

> amount of time it takes to move it, also known as "bandwidth".

It's not that large frames hurt multiplexing necessarily, it's that it
hurts "responsiveness."

If you are in the process of sending a large data frame, you cannot
respond to a PRIORITY frame that prioritizes a different stream and
begin to send data for that stream until the entire frame is flushed.

---

From: Michael Sweet <[email-masked]>
Date: Tue, 08 Jul 2014 13:01:23 -0400

Jeff,

> begin to send data for that stream until the entire frame is flushed.

How fast do you need to respond to a PRIORITY frame?  What are you trying to do that requires such a sudden priority change? What are your performance requirements?

Let's take a 5mbit/sec DSL connection (pretty typical low-end "high speed" service offered in many locations).  The time to transmit a 16k DATA frame over that link would be approximately 33ms, 32k would take ~66ms, 64k would take ~131ms, and so forth.  None of those times seem overly long to me, and certainly I'm not going to really notice any loss of responsiveness if the page or new video starts loading 1/8th of a second after I click on a link.

But what I *will* notice is video stuttering caused by too much multiplexing, or apparently hung page loads because the priority for a video stream was set to the maximum just to ensure that two small DATA frames would be delivered close together so that the video doesn't stutter.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 10:04:21 -0700

> How fast do you need to respond to a PRIORITY frame?  What are you trying to do that requires such a sudden priority change? What are your performance requirements?

FWIW, my suggestion has been to increase the frame length field to the
full 16 bits. Applications that care about interactivity (mostly web
browsing on low-latency networks) can choose to send smaller data
payloads (say 8 kb)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 08 Jul 2014 16:37:13 +0000

In message <CAPyZ6=KnoDue9de0qg=nqGnsgkJmCYBzu1NMqbxpAN39P7rw=[email-masked]>, Tatsuhiro Tsujik
awa writes:

>Multiple DATA frames can consume large bandwidth too and thereby deliver
>large data quickly.

And they will have multiple the overhead of a single larger frame.

>So the goal of HTTP/2 is to design it for low power embedded devices?

The goal, as I understood it, is to design a protocol that will be
generally usable.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 08 Jul 2014 16:40:51 +0000

In message <[email-masked]>, Jeff Pinner writ
>It's not that large frames hurt multiplexing necessarily, it's that it

I assumed that was TT meant by "multiplexing".

The point here is that nobody forces TT to neither send nor receive
frames larger than 16K and that he should extend the same courtesy
to people who want to do so.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 8 Jul 2014 10:43:45 -0700

> constrained by a setting.

Not really.  The math is pretty simple:

uncompressed_size[max] = compressed_size * header_table_size / 2

So yes, constrained, but not really reasonable.

---

From: <[email-masked]>
Date: Tue, 8 Jul 2014 18:02:35 +0000

On Tuesday,08 July 2014 19:44, [email-masked] wrote:
>> indexed ones, and they reference memory in the header set that is
>> already constrained by a setting.
>

Solve for compressed_size...

compressed_size = uncompressed_size[max] * 2 / header_table_size

And that's your setting.  I must be missing your point?? (An endpoint should know how much it wants to commit for the uncompressed data and how much it wants to commit for the table.)

>So yes, constrained, but not really reasonable.

Can you please clarify what's not reasonable?  Not a reasonable estimation? Or, not reasonable to commit that much memory?

---

From: Martin Thomson <[email-masked]>
Date: Tue, 8 Jul 2014 11:15:05 -0700

> And that's your setting.  I must be missing your point?? (An endpoint should know how much it wants to commit for the uncompressed data and how much it wants to commit for the table.)

WTF?  That would be completely nonsensical.  The formula only produces
an upper bound, not a sensible value.

Actually, the more complete formula is slightly worse with Huffman
encoding and the reference set:

uncompressed_size[max] = (compressed_size + 1) * (header_table_size -
32) / 2 / (5/8)

> Can you please clarify what's not reasonable?  Not a reasonable estimation? Or, not reasonable to commit that much memory?

Let's say that you commit a 4k header table and permit frames of 16k
(i.e., the default values we're talking about).  That means that you
are potentially required to commit ~32M for an uncompressed set of
header fields that exploit the maximum compression (actually, it's a
little higher than this, but you get the idea).

---

From: <[email-masked]>
Date: Tue, 8 Jul 2014 19:52:42 +0000

> On 08 Jul 2014, at 20:15, "[email-masked]" <[email-masked]> wrote:
>

Yup. Totally missed your point. Got it.

> little higher than this, but you get the idea).

This part I still don't get. What does it matter if the uncompressed headers are 32M, or 32G for that matter? As soon as you reach the limit you are willing to commit in resources to the uncompressed headers, you'll stop and respond with 431.  Of course you'll have to decide if you should keep processing to synchronize state, but if a portion of 16K is generating MB uncompressed, it should be pretty obvious you are under a DoS attack and kill the connection.  Am I still missing something?

---

From: Martin Thomson <[email-masked]>
Date: Tue, 8 Jul 2014 13:46:46 -0700

On 8 July 2014 12:52,  <[email-masked]> wrote:
> What does it matter if the uncompressed headers are 32M, or 32G for that matter? As soon as you reach the limit you are willing to commit in resources to the uncompressed headers, you'll stop and respond with 431.

Correct.  And I'm merely noting that having a setting does nothing to
help you avoid having to follow that process.  The setting - on its
own - is therefore of marginal value.  Mark was suggesting that we
consider the setting in isolation, which is the point that I was
addressing.

---

From: Jason Greene <[email-masked]>
Date: Tue, 8 Jul 2014 17:09:40 -0500

> consider the setting in isolation, which is the point that I was

Just one minor caveat I feel like making. Technically a lot of this depends on the compute model of the server. It’s possible, for example, to take advantage of the HPACK referential model. N references to the header table do not necessarily mean N * entry size. It’s quite possible that a memory optimized implementation would have a formula which was (compressed size * sizeof(void *)) + header_size. Or perhaps even compressed size + header_size if it simply mirrored the compact HPACK structure in memory.

Although to your point, that’s not something that can be relied upon.
--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat