Subject: Large Frame Proposal

From: Greg Wilkins <[email-masked]>
Date: Mon, 7 Jul 2014 17:50:55 +1000

https://github.com/http2/http2-spec/pull/548

 Background

The HTTP2 protocol has a requirement to be able to transport large headers,
that exceed the payload size of a single frame at the current 16KB maximum
size.

To address this requirement, the current draft (13) includes the
CONTINUATION frames, 0 or more of which may be sent after a HEADERS or
PUSH_PROMISE frame to contain the large headers. There has been significant
criticism of the CONTINUATION design, including:

   -

   The total length of a HEADERS+CONTINUATION* sequence is not known until
   the last frame in the sequence is processed. A receiver that wishes to
   reject streams headers larger than a specific limit may have to process
   many frames and hold the results in memory before it discovers the header
   is too large.
   -

   The size of header that an endpoint is prepared to receive is not known
   in advance. The only way a sender can know if a header too large is by
   attempting to send it and receiving an error in response. Error handling of
   headers may be difficult for an endpoint to handle efficiently and can
   result in the closure of the entire connection.
   -

   The END_STREAM flag is not present on the CONTINUATION frame, thus it is
   possible for a stream to send CONTINUATION frames after a HEADERS frame
   that has the END_STREAM flag set. This is confusing and increases the
   complexity of the state machine required to process streams. It is highly
   desirable that a set END_STREAM flag truly indicates the last non control
   frame of a stream.
   -

   There is a significant discontinuity in the code path required to
   process headers. Headers up to an indeterminate size (roughly 20-something
   KB) can be handled in a single frame. Headers that exceed this size must be
   handled in multiple frames of different types with different frame flags
   and stream control logic. Because the vast majority of headers sent
   (>99.99%) are below this indeterminate size, implementations will have a
   code path that is seldom executed and probably insufficiently tested. This
   invites poor and/or partial and/or incorrect implementation.
   -

   Because of the HPACK compression algorithm, a sequence of
   HEADERS+CONTINUATIONS frames may not be interleaved with any other frame.
   This effectively makes the sequence a single large frame. Because of the
   simplicity of description and implementation it is proposed that it would
   be far simpler to meet the requirement of large headers by supporting large
   frames.

This proposal has been prepared as it is possible to meet the requirements
of CONTINUATIONS without the complications and criticisms above.

This proposal addresses the issue of sending/receiving large HTTP headers
without giving endpoints and intermediaries unlimited resource commitments
nor unknown limits

Additional Frame Size Issues Addressed

The current draft (13) has maximum frame size of 16KB, which is an
arbitrary value that has been selected on the basis of experience to
provide a reasonable compromise between the efficiency of transmitting data
vs the quality of service for multiplexed channels.

Whilst this educated guess may be near optimal for today's networks and
traffic, it is entirely possible that some current and/or future networks
may require a different value to achieve an optimal balance. There have
already been proposals [1] put to the WG to reduce the frame size to
optimise multiplexing , as well as discussion that high capacity, low
latency networks can achieve satisfactory multiplexing quality of service
with large frame sizes.

This proposal addresses the issue that a fixed frame size does not allow
tuning multiplexing performance based on current/future experience.

It has also been noted that 16KB is near the middle of the peak of the
current HTTP Object size histogram [2], so that a small change in the frame
size may have a significant impact on the number of HTTP messages that can
be sent in a single frame, without significant impacts on QoS. The HTTP
Object size histogram has changed significantly over time and is expected
to continue to do so.

This proposal addresses the issue of tuning the frame size based on
experience of actual payload sizes.

There have also been issues raised that a 16KB frame size does not allow
efficient data transfer [3] even when the end points are aware that only a
single stream is likely to be required for the imminent future, or
that a particular stream is of high priority.

This proposal addresses the issue of tuning the frame size for transport
efficiency for specific streams in specific situations.

Large Frame Header Proposal

This proposal is to alter the core http2 protocol to address the issues
identified above by supporting a variable length maximum frame size
controlled by peer limits.

This proposal increases the length field in the frame header to 31 bits, to
match the maximum flow control window size. However, implementations will
not be able to use the full frame size without explicit consent from peers
using newly defined SETTINGS or an optional WINDOW_UPDATE field.
Frame Size Settings

Two settings parameters are proposed: SETTINGS_HEADER_FRAME_SIZE for the
maximum header size and SETTINGS_FRAME_SIZE for all other frames.

The SETTINGS_HEADER_FRAME_SIZE parameter supports the current behaviour
where large headers can be sent without changing the frame size allowed for
other frame types. ie A large header size limit can be set without
affecting the multiplexing efficiency of DATA frames.

The SETTINGS_FRAME_SIZE applies to all other frames including DATA frames
and any other frame that may be defined by an extension. The use of this
parameter is intended to tune/optimise the connection for the general case
of multiple streams over the specific connection.
Frame Size Updates

To handle the issue of efficiently sending large data when an end point is
prepared to risk multiplexing efficiency, this proposal allows a Max Frame
Size to be applied to a specific stream as an optional field in a
WINDOW_UPDATE frame.

By including a variable frame size in the flow control mechanism this
proposal allows the decision to increase the frame size to be deferred
until more knowledge about the specific situation are known and limited to
the stream that will benefit from the increased size.

Consider the example of a server that has commenced sending a large content
to a client. The server may initially send 4 x 16KB frames to consume the
default stream flow control window, at which time it must wait for the
client to send a WINDOW_UPDATE frame before continuing. When generating the
WINDOW_UPDATE frame, the client may have knowledge of:

   - The content-length header - so it knows that the amount of data
   expected is large or is perhaps just slightly larger than a single frame.
   - The content-type header - so it knows if the content has high priority
   in rendering the current page, or if the content is likely to include
   references to other resources which may need to be multiplexed.
   - How many other streams are currently open and/or reserved - so it
   knows if multiplexing is actually required.
   - How many other requests are pending - so it knows if new multiplexed
   stream could soon be opened.
   - An approximate rough measure of the network latency and throughput -
   This can be derived from the timing of the receipt of the first few data
   frames and used to estimate the impact on QoS of any change to the maximum
   frame size.

The client can use this knowledge to make an informed decision as to the
benefit of changing the allowed max frame size against any risk to
multiplexing QoS. It can make several choices:

   -

   No change. It can just not adjust the frame size, either because it is
   too hard to consider or that there are too many other streams, or that the
   content is video that needs to be received slowly. In any of these cases
   the max frame size can be left unchanged and the protocol continues as it
   currently does.
   -

   Large frame. If the stream is the only expected stream or of
   sufficiently high priortiy, then the window and frame size can be set to
   allow as much of the remaining data as possible to be sent in a single
   frame.
   -

   Medium frame. The client can momentarily trade some QoS (for an
   estimated duration) by increasing the frame size to something >16KB and <
   content-length
   -

   Sufficient frame. If the remaining content is only a small increment
   over the current SETTING_FRAME_SIZE, the Max Frame Size can be increased to
   receive the remaining content in a single frame without any significant QoS
   impact.

Minimal Compliance

A minimally compliant implementation MUST handle the SETTING_FRAME_SIZE and
SETTINGS_HEADER_SIZE and ensure that no frame sent exceeds the applicable
limit. However no implementation is required to send frames at or near
these limits when set above the default 16KB.

There is no requirement for an implementation to send or to handle the Max
Frame Size in a WINDOW_UPDATE and it is allowable for it to be ignored if
received.
Anticipated Feedback

*It is too late in the process to change the framing layer and to do so
after so much discussion is an implicit fail of the WG*

To not consider issues and proposal brought to the WG would be a fail of
the process. This proposal is based on all the hard work to date done by
the WG and contributors to identify issues and test solutions.

*These issues can be handled in extensions.*

Optimising data transfers for large content could possibly be done in an
extension, however:

   - It is not yet clear if extensions will be a viable way to enhance the
   http2 protocol. There are significant hurdle to overcome to deploy
   extensions.
   - Many of the issues are aimed at complexity and tuning of the core
   protocol, and these cannot be addressed in an extension.
   - It is asymmetric to support large headers with one mechanism and large
   data with another.

*The proposed header costs 2 extra bytes per frame*

There is a small data cost to adopt this proposal, however this is
mitigated as:

   - The proposal may be able to reduce the number of frames needed for
   some content, thus saving 8 bytes. Whilst not likely to be a 25% frame
   saving required to break even, it will still reduce cost to below 2 bytes.
   - There are options to have variable length headers or optional extended
   headers that will preserve the semantics of this proposal and keep an 8
   byte header for small frames. If the 2 byte cost is considered prohibitive,
   then these alterations can be considered.

*The header is 10 bytes long and not 32bit word aligned.*

Frames sent after arbitrary data will not be word aligned anyway. If
alignment is important, then padding could changed to be part of the base
frame format, 2 header bytes used for a padding length (giving an aligned
12 byte header) and all frames padded to a word boundary.

 *31 bits is also an arbitrary length*

It is true that a 31 bit large frame length is also an arbitrary limit to
the size of a frame. However, it is believed that 31 bits is sufficiently
large to efficiently handle almost all conceivable present and future use
cases. It would be possible to implement an unlimited size length field,
but this would also need changes to the flow control mechanism, which
currently also has a 31 bit maximum size.

 *It does not support unlimited response headers*

A SETTINGS_HEADER_FRAME_SIZE of 2^31-1 is effectively unlimited for all
foreseeable response headers.

 *This was tried with SPDY and rejected*

SPDY did not have the settings to allow peers to set limits on the max
frame size. This proposal will not change to default behaviour of http2
with regards to frame size.

 *Intermediaries will destroy multiplexing by setting frame size to 2^31-1*

Large frames require the participation of both sender and receiver. A
receiver may advise that it is willing to accept large frames, but a sender
is under no obligation to send them. Thus intermediaries nor any end point
can unilaterally change multiplexing QoS.

 *Cannot be hardware accelerated.*

Hardware acceleration is not part of the WG brief to support, nor is it
clear that this proposal is any less suitable than others for hardware
acceleration.
Contributors

This proposal was prepare by:

   - Amos Jeffries [email-masked]
   - Greg Wilkins [email-masked]
   - Jason Greene [email-masked]
   - Keith Morgan [email-masked]
   - Poul-Henning Kamp [email-masked]

[1] http://lists.w3.org/Archives/Public/ietf-http-wg/2013AprJun/0926.html
[2] http://httparchive.org/interesting.php
[3] http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1664.html

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 7 Jul 2014 18:27:16 +1000

Hi Greg (et al),

Thanks for the proposal. The most immediate concern I have here is how SETTINGS is used; see my recent e-mail to Jason.

Cheers,

On 7 Jul 2014, at 5:50 pm, Greg Wilkins <[email-masked]> wrote:

> The HTTP2 protocol has a requirement to be able to transport large headers, that exceed the payload size of a single frame at the current 16KB maximum size.
> 
> To address this requirement, the current draft (13) includes the CONTINUATION frames, 0 or more of which may be sent after a HEADERS or PUSH_PROMISE frame to contain the large headers. There has been significant criticism of the CONTINUATION design, including:
> 
> 	• The total length of a HEADERS+CONTINUATION* sequence is not known until the last frame in the sequence is processed. A receiver that  wishes to reject streams headers larger than a specific limit may have to process many frames and hold the results in memory before it discovers the header is too large.
> 
> 	• The size of header that an endpoint is prepared to receive is not known in advance. The only way a sender can know if a header too large is by attempting to send it and receiving an error in response. Error handling of headers may be difficult for an endpoint to handle efficiently and can result in the closure of the entire connection.
> 
> 	• The END_STREAM flag is not present on the CONTINUATION frame, thus it is possible for a stream to send CONTINUATION frames after a HEADERS frame that has the END_STREAM flag set. This is confusing and increases the complexity of the state machine required to process streams. It is highly desirable that a set END_STREAM flag truly indicates the last non control frame of a stream.
> 
> 	• There is a significant discontinuity in the code path required to process headers. Headers up to an indeterminate size (roughly 20-something KB) can be handled in a single frame. Headers that exceed this size must be handled in multiple frames of different types with different frame flags and stream control logic. Because the vast majority of headers sent (>99.99%) are below this indeterminate size, implementations will have a code path that is seldom executed and probably insufficiently tested. This invites poor and/or partial and/or incorrect implementation.
> 
> 	• Because of the HPACK compression algorithm, a sequence of HEADERS+CONTINUATIONS frames may not be interleaved with any other  frame. This effectively makes the sequence a single large frame. Because of the simplicity of description and implementation it is proposed that it would be far simpler to meet the requirement of large headers by supporting large frames.
> 
> This proposal has been prepared as it is possible to meet the requirements of CONTINUATIONS without the complications and criticisms above.
> 
> This proposal addresses the issue of sending/receiving large HTTP headers without giving endpoints and intermediaries unlimited resource commitments nor unknown limits
> 
> The current draft (13) has maximum frame size of 16KB, which is an arbitrary value that has been selected on the basis of experience to provide a reasonable compromise between the efficiency of transmitting data vs the quality of service for multiplexed channels.
> 
> Whilst this educated guess may be near optimal for today's networks and traffic, it is entirely possible that some current and/or future networks may require a different value to achieve an optimal balance. There have already been proposals [1] put to the WG to reduce the frame size to optimise multiplexing , as well as discussion that high capacity, low latency networks can achieve satisfactory multiplexing quality of service with large frame sizes.
> 
> This proposal addresses the issue that a fixed frame size does not allow tuning multiplexing performance based on current/future experience.
> 
> It has also been noted that 16KB is near the middle of the peak of the current HTTP Object size histogram [2], so that a small change in the frame size may have a significant impact on the number of HTTP messages that can be sent in a single frame, without significant impacts on QoS. The HTTP Object size histogram has changed significantly over time and is expected to continue to do so.
> 
> This proposal addresses the issue of tuning the frame size based on experience of actual payload sizes.
> 
> There have also been issues raised that a 16KB frame size does not allow efficient data transfer [3] even when the end points are aware that only a single stream is likely to be required for the imminent future, or
> This proposal addresses the issue of tuning the frame size for transport efficiency for specific streams in specific situations.
> 
> This proposal is to alter the core http2 protocol to address the issues identified above by supporting a variable length maximum frame size controlled by peer limits.
> 
> This proposal increases the length field in the frame header to 31 bits, to match the maximum flow control window size. However, implementations will not be able to use the full frame size without explicit consent from peers using newly defined SETTINGS or an optional WINDOW_UPDATE field.
> 
> Frame Size Settings
> 
> Two settings parameters are proposed: SETTINGS_HEADER_FRAME_SIZE for the maximum header size and SETTINGS_FRAME_SIZE for all other frames.
> 
> The SETTINGS_HEADER_FRAME_SIZE parameter supports the current behaviour where large headers can be sent without changing the frame size allowed for other frame types. ie A large header size limit can be set without affecting the multiplexing efficiency of DATA frames.
> 
> The SETTINGS_FRAME_SIZE applies to all other frames including DATA frames and any other frame that may be defined by an extension. The use of this parameter is intended to tune/optimise the connection for the general case of multiple streams over the specific connection.
> 
> Frame Size Updates
> 
> To handle the issue of efficiently sending large data when an end point is prepared to risk multiplexing efficiency, this proposal allows a Max Frame Size to be applied to a specific stream as an optional field in a WINDOW_UPDATE frame.
> 
> By including a variable frame size in the flow control mechanism this proposal allows the decision to increase the frame size to be deferred until more knowledge about the specific situation are known and limited to the stream that will benefit from the increased size.
> 
> Consider the example of a server that has commenced sending a large content to a client. The server may initially send 4 x 16KB frames to consume the default stream flow control window, at which time it must wait for the client to send a WINDOW_UPDATE frame before continuing. When generating the WINDOW_UPDATE frame, the client may have knowledge of:
> 
> 	• The content-length header - so it knows that the amount of data expected is large or is perhaps just slightly larger than a single frame.
> 	• The content-type header - so it knows if the content has high priority in rendering the current page, or if the content is likely to include references to other resources which may need to be multiplexed.
> 	• How many other streams are currently open and/or reserved - so it knows if multiplexing is actually required.
> 	• How many other requests are pending - so it knows if new multiplexed stream could soon be opened.
> 	• An approximate rough measure of the network latency and throughput - This can be derived from the timing of the receipt of the first few data frames and used to estimate the impact on QoS of any change to the maximum frame size.
> The client can use this knowledge to make an informed decision as to the benefit of changing the allowed max frame size against any risk to multiplexing QoS. It can make several choices:
> 
> 	• No change. It can just not adjust the frame size, either because it is too hard to consider or that there are too many other streams, or that the content is video that needs to be received slowly. In any of these cases the max frame size can be left unchanged and the protocol continues as it currently does.
> 
> 	• Large frame. If the stream is the only expected stream or of sufficiently high priortiy, then the window and frame size can be set to allow as much of the remaining data as possible to be sent in a single frame.
> 
> 	• Medium frame. The client can momentarily trade some QoS (for an estimated duration) by increasing the frame size to something >16KB and < content-length
> 
> 	• Sufficient frame. If the remaining content is only a small increment over the current SETTING_FRAME_SIZE, the Max Frame Size can be increased to receive the remaining content in a single frame without any significant QoS impact.
> 
> Minimal Compliance
> 
> A minimally compliant implementation MUST handle the SETTING_FRAME_SIZE and SETTINGS_HEADER_SIZE and ensure that no frame sent exceeds the applicable limit. However no implementation is required to send frames at or near these limits when set above the default 16KB.
> 
> There is no requirement for an implementation to send or to handle the Max Frame Size in a WINDOW_UPDATE and it is allowable for it to be ignored if received.
> 
> Anticipated Feedback
> 
> It is too late in the process to change the framing layer and to do so after so much discussion is an implicit fail of the WG
> 
> To not consider issues and proposal brought to the WG would be a fail of the process. This proposal is based on all the hard work to date done by the WG and contributors to identify issues and test solutions.
> 
> These issues can be handled in extensions.
> 
> Optimising data transfers for large content could possibly be done in an extension, however:
> 
> 	• It is not yet clear if extensions will be a viable way to enhance the http2 protocol. There are significant hurdle to overcome to deploy extensions.
> 	• Many of the issues are aimed at complexity and tuning of the core protocol, and these cannot be addressed in an extension.
> 	• It is asymmetric to support large headers with one mechanism and large data with another.
> 
> 
> The proposed header costs 2 extra bytes per frame
> 
> There is a small data cost to adopt this proposal, however this is mitigated as:
> 
> 	• The proposal may be able to reduce the number of frames needed for some content, thus saving 8 bytes. Whilst not likely to be a 25% frame saving required to break even, it will still reduce cost to below 2 bytes.
> 	• There are options to have variable length headers or optional extended headers that will preserve the semantics of this proposal and keep an 8 byte header for small frames. If the 2 byte cost is considered prohibitive, then these alterations can be considered.
> 
> 
> The header is 10 bytes long and not 32bit word aligned.
> 
> Frames sent after arbitrary data will not be word aligned anyway. If alignment is important, then padding could changed to be part of the base frame format, 2 header bytes used for a padding length (giving an aligned 12 byte header) and all frames padded to a word boundary.
> 
> 
> 31 bits is also an arbitrary length
> 
> It is true that a 31 bit large frame length is also an arbitrary limit to the size of a frame. However, it is believed that 31 bits is sufficiently large to efficiently handle almost all conceivable present and future use cases. It would be possible to implement an unlimited size length field, but this would also need changes to the flow control mechanism, which currently also has a 31 bit maximum size.
> 
> 
> It does not support unlimited response headers
> 
> A SETTINGS_HEADER_FRAME_SIZE of 2^31-1 is effectively unlimited for all foreseeable response headers.
> 
> 
> 
> This was tried with SPDY and rejected
> 
> SPDY did not have the settings to allow peers to set limits on the max frame size. This proposal will not change to default behaviour of http2 with regards to frame size.
> 
> 
> 
> Intermediaries will destroy multiplexing by setting frame size to 2^31-1
> 
> Large frames require the participation of both sender and receiver. A receiver may advise that it is willing to accept large frames, but a sender is under no obligation to send them. Thus intermediaries nor any end point can unilaterally change multiplexing QoS.
> 
> 
> 
> Cannot be hardware accelerated.
> 
> Hardware acceleration is not part of the WG brief to support, nor is it clear that this proposal is any less suitable than others for hardware acceleration.
> 
> Contributors
> 
> 	• Amos Jeffries [email-masked]
> 	• Greg Wilkins [email-masked]
> 	• Jason Greene [email-masked]
> 	• Keith Morgan [email-masked]
> 	• Poul-Henning Kamp [email-masked]

--
Mark Nottingham   https://www.mnot.net/

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 7 Jul 2014 18:39:30 +1000

On 7 July 2014 18:27, Mark Nottingham <[email-masked]> wrote:

> Hi Greg (et al),
>
> Thanks for the proposal. The most immediate concern I have here is how
> SETTINGS is used; see my recent e-mail to Jason.
>
>
Mark,

In reference to your comment:

> SETTINGS is explicitly not a negotiation mechanism; it only allows one
end to advertise its configuration to its peer.
>
> In this case, the semantic of MHS (MAX_HEADER_SIZE) would be roughly "I
allow headers
> coming at me to be at most THIS BIG." There's no way for the peer to ask
for more; it has to
> accept the limitation.

Correct!  This is not intended to be a negotiation, it is simply intended
to make explicit the current HTTP semantics that allows a server to reject
large entities.    It is not an invitation to haggle with an endpoint, the
endpoint is simply saying - don't even think about sending me a header
larger than this setting.

Endpoint already have limitation, which can only be discovered by
attempting to send a header block, only to receive a 413.  This proposal
puts the burden on the sender rather than the receiver to enforce such
limitations.

cheers

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 7 Jul 2014 20:12:45 +1000

Thanks, Greg.

I'm very much of two minds about this proposal.

On the one hand, it's very late in the process; if these issues and this proposal had been articulated six or even three months ago, we'd be having a very different conversation. We committed at our recent meeting to go to WGLC on a tight schedule, and it's not at all certain that such an invasive change is necessary to resolve the underlying issues. As such, I suspect implementer enthusiasm is going to be dim (and yes, I saw the attempt to address this in the write-up).

On the other hand, it is a very interesting proposal, and it does seem to address many potential objections as well as the underlying issues. As such, I want to see the Working Group take it seriously; we cannot simply ignore it, if only because doing so is very likely to get us an even larger delay in our LCs and beyond.

I'm currently discussing this with our AD to make sure we get the balance right.

Stepping back, I'll make one observation -- we're seeing a lot of proposals, and I think the underlying issues are still muddy. I've tried to separate them out in the issues list; see:
   https://github.com/http2/http2-spec/issues/549
   https://github.com/http2/http2-spec/issues/550
   https://github.com/http2/http2-spec/issues/551

If we can't gain consensus one of our current proposals, we're still going to need to have good answers for each of these issues.

Regards,

P.S. I suspect that the relative silence from many over the last few days is due to the US long weekend; welcome back,and please get into the discussion ASAP.

On 7 Jul 2014, at 6:39 pm, Greg Wilkins <[email-masked]> wrote:

> > SETTINGS is explicitly not a negotiation mechanism; it only allows one end to advertise its configuration to its peer.
> >
> > In this case, the semantic of MHS (MAX_HEADER_SIZE) would be roughly "I allow headers 
> > coming at me to be at most THIS BIG." There's no way for the peer to ask for more; it has to 
> > accept the limitation.
>  
> Correct!  This is not intended to be a negotiation, it is simply intended to make explicit the current HTTP semantics that allows a server to reject large entities.    It is not an invitation to haggle with an endpoint, the endpoint is simply saying - don't even think about sending me a header larger than this setting.
> 
> Endpoint already have limitation, which can only be discovered by attempting to send a header block, only to receive a 413.  This proposal puts the burden on the sender rather than the receiver to enforce such limitations.
> 
> cheers
> 
> 
> 
> 
> 
> 
> --

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 7 Jul 2014 21:44:22 +1000

Mark,

thanks for the serious consideration.

I agree that the issues need to be carefully identified.   I think 549,550
and 551 are all good issues to create (although this proposal does not
directly address 550).   But I think there should also be issues for

   - a fixed frame size does not allow tuning multiplexing performance
   based on current/future experience.
   - the 16KB frame size is not well tuned for frequent payload sizes
   - a fixed frame size cannot be adjusted for specific streams in specific
   situations that may not required multiplexing efficiency.

Or at least one issue that the fixed max frame size cannot be tuned for any
reason

One note of caution with decomposing to individual issues, is that no
single issue may be sufficiently important to rock the boat at this late
state, but a proposal that resolves many such issues might well be worth
the disruption.

cheers

On 7 July 2014 20:12, Mark Nottingham <[email-masked]> wrote:

> On the one hand, it's very late in the process; if these issues and this
> proposal had been articulated six or even three months ago, we'd be having
> a very different conversation. We committed at our recent meeting to go to
> WGLC on a tight schedule, and it's not at all certain that such an invasive
> change is necessary to resolve the underlying issues. As such, I suspect
> implementer enthusiasm is going to be dim (and yes, I saw the attempt to
> address this in the write-up).
>
> On the other hand, it is a very interesting proposal, and it does seem to
> address many potential objections as well as the underlying issues. As
> such, I want to see the Working Group take it seriously; we cannot simply
> ignore it, if only because doing so is very likely to get us an even larger
> delay in our LCs and beyond.
>
> Stepping back, I'll make one observation -- we're seeing a lot of
> proposals, and I think the underlying issues are still muddy. I've tried to
> separate them out in the issues list; see:
> If we can't gain consensus one of our current proposals, we're still going
> to need to have good answers for each of these issues.
>
> Regards,
>
> P.S. I suspect that the relative silence from many over the last few days
> is due to the US long weekend; welcome back,and please get into the
> discussion ASAP.
>
>
> > Correct!  This is not intended to be a negotiation, it is simply
> intended to make explicit the current HTTP semantics that allows a server
> to reject large entities.    It is not an invitation to haggle with an
> endpoint, the endpoint is simply saying - don't even think about sending me
> a header larger than this setting.
> >

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 11:41:21 +0000

In message <[email-masked]>, Mark Nottingham writes:

>We committed at our recent meeting to go to WGLC on a tight schedule,

I may have missed it, but for those of us not present, could you summarize
what the rush is ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Michael Sweet <[email-masked]>
Date: Mon, 07 Jul 2014 09:13:16 -0400

Mark,

Speaking as an (almost completed) implementor, this proposal solves my issues with needing to implement CONTINUATION frames just to support Kerberos authentication - servers that need Kerberos authentication can advertise their ability to receive larger header frames; clients that can do Kerberos will already have the resources to send them.

It also solves an issue I haven't yet reported (mainly because I don't have any evidence for HTTP/2 yet) where many current printers benefit from larger frame sizes.  In current HTTP/1.1 implementations, going from a maximum chunk size of 16k to 64k has yielded a 2x performance improvement (over Wi-Fi or Gigabit Ethernet).  The overhead of processing chunks in a low-resource, embedded processor is fairly high, so anything we can do to tune the chunk/frame size to the maximum the printer can safely handle (i.e. avoid a deadlock situation that prevents our reporting a paper jam or other condition to the user) will make for a better user experience.

>> Endpoint already have limitation, which can only be discovered by attempting to send a header block, only to receive a 413.  This proposal puts the burden on the sender rather than the receiver to…

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Martin Thomson <[email-masked]>
Date: Mon, 7 Jul 2014 10:42:41 -0700

I think that this proposal has some merit.  I, like Mark, am concerned
that it will get a negative reaction from people who want to ship
their existing code and are not sufficiently motivated to change.

Assuming now that the general concept is acceptable...

My one big reservation is with the addition to WINDOW_UPDATE.  I think
that it's unnecessary complexity.  The other changes you make are
well-enough supported, but this seems like feature creep to me.
Having to track a limit on each stream is more state and complexity.
And making it optional and ignorable means that you've created an
optional feature that is optional, which is a poor way to guarantee
that implementations will provide and respect it.

I think that the setting for maximum frame size needs to be just
limited to DATA.

I think that the setting for maximum header block size needs to cover
PUSH_PROMISE (we already have settings that govern PUSH_PROMISE).

Any why 16384 rather than 16383?

There are a few editorial problems with the pull request, but nothing
major.  The edit to the header block section seems to have been
rushed; the minimum values on size need to be better called out.

On 7 July 2014 00:50, Greg Wilkins <[email-masked]> wrote:
> The total length of a HEADERS+CONTINUATION* sequence is not known until the
> last frame in the sequence is processed. A receiver that wishes to reject
> streams headers larger than a specific limit may have to process many frames
> and hold the results in memory before it discovers the header is too large.
>
> The size of header that an endpoint is prepared to receive is not known in
> advance. The only way a sender can know if a header too large is by
> possible for a stream to send CONTINUATION frames after a HEADERS frame that
> has the END_STREAM flag set. This is confusing and increases the complexity
> of the state machine required to process streams. It is highly desirable
> that a set END_STREAM flag truly indicates the last non control frame of a
> stream.
>
> There is a significant discontinuity in the code path required to process
> headers. Headers up to an indeterminate size (roughly 20-something KB) can
> be handled in a single frame. Headers that exceed this size must be handled
> in multiple frames of different types with different frame flags and stream
> control logic. Because the vast majority of headers sent (>99.99%) are below
> this indeterminate size, implementations will have a code path that is
> seldom executed and probably insufficiently tested. This invites poor and/or
> partial and/or incorrect implementation.
>
> simplicity of description and implementation it is proposed that it would be
> far simpler to meet the requirement of large headers by supporting large
> frames.
>
> The current draft (13) has maximum frame size of 16KB, which is an arbitrary
> value that has been selected on the basis of experience to provide a
> reasonable compromise between the efficiency of transmitting data vs the
> quality of service for multiplexed channels.
>
> Object size histogram has changed significantly over time and is expected to
> continue to do so.
>
> other frame types. ie A large header size limit can be set without affecting
> the multiplexing efficiency of DATA frames.
>
> proposal allows the decision to increase the frame size to be deferred until
> more knowledge about the specific situation are known and limited to the
> stream that will benefit from the increased size.
>
> The content-length header - so it knows that the amount of data expected is
> large or is perhaps just slightly larger than a single frame.
> The content-type header - so it knows if the content has high priority in
> rendering the current page, or if the content is likely to include
> How many other streams are currently open and/or reserved - so it knows if
> multiplexing is actually required.
> How many other requests are pending - so it knows if new multiplexed stream
> could soon be opened.
> An approximate rough measure of the network latency and throughput - This
> can be derived from the timing of the receipt of the first few data frames
> and used to estimate the impact on QoS of any change to the maximum frame
> size.
>
> No change. It can just not adjust the frame size, either because it is too
> hard to consider or that there are too many other streams, or that the
> content is video that needs to be received slowly. In any of these cases the
> max frame size can be left unchanged and the protocol continues as it
> currently does.
>
> Large frame. If the stream is the only expected stream or of sufficiently
> high priortiy, then the window and frame size can be set to allow as much of
> the remaining data as possible to be sent in a single frame.
>
> Medium frame. The client can momentarily trade some QoS (for an estimated
> duration) by increasing the frame size to something >16KB and <
> content-length
>
> Sufficient frame. If the remaining content is only a small increment over
> the current SETTING_FRAME_SIZE, the Max Frame Size can be increased to
> limit. However no implementation is required to send frames at or near these
> limits when set above the default 16KB.
>
> It is too late in the process to change the framing layer and to do so after
> so much discussion is an implicit fail of the WG
>
> To not consider issues and proposal brought to the WG would be a fail of the
> process. This proposal is based on all the hard work to date done by the WG
> and contributors to identify issues and test solutions.
>
> It is not yet clear if extensions will be a viable way to enhance the http2
> protocol. There are significant hurdle to overcome to deploy extensions.
> Many of the issues are aimed at complexity and tuning of the core protocol,
> and these cannot be addressed in an extension.
> It is asymmetric to support large headers with one mechanism and large data
> with another.
>
>
> The proposal may be able to reduce the number of frames needed for some
> content, thus saving 8 bytes. Whilst not likely to be a 25% frame saving
> required to break even, it will still reduce cost to below 2 bytes.
> There are options to have variable length headers or optional extended
> headers that will preserve the semantics of this proposal and keep an 8 byte
> header for small frames. If the 2 byte cost is considered prohibitive, then
> these alterations can be considered.
>
>
> frame format, 2 header bytes used for a padding length (giving an aligned 12
> byte header) and all frames padded to a word boundary.
>
> cases. It would be possible to implement an unlimited size length field, but
> this would also need changes to the flow control mechanism, which currently
> also has a 31 bit maximum size.
>
> SPDY did not have the settings to allow peers to set limits on the max frame
> size. This proposal will not change to default behaviour of http2 with
> regards to frame size.
>
>
> Amos Jeffries [email-masked]
> Greg Wilkins [email-masked]
> Jason Greene [email-masked]
> Keith Morgan [email-masked]
> Poul-Henning Kamp [email-masked]
>

---

From: Nicholas Hurley <[email-masked]>
Date: Mon, 7 Jul 2014 10:53:06 -0700

So now we're resurrecting jumbo frames. Again. I've already said "no" to
this, and apparently I have to say it again. No jumbo frames, no large
frames. Anyone who cares to see my reasoning against
jumbo/large/huge/whatever frames is welcome to go back and read the
archives.

This is a HUGE change to the spec in order to make people feel better about
cases that make up 0.02% of the streams out there. Why are we continuing to
spend time and energy on this?

One of the problems this tries to address -- END_STREAM on HEADERS vs.
CONTINUATION -- I'd be fine with changing in the existing spec. Put the
flag wherever.

If the WG decides to go this route (which I think would be ill-advised),
there NEEDS to be a minimum on max frame size. We don't need to provide
people two directions (too low and too high) to shoot themselves in the
foot with.

Finally, this doesn't ACTUALLY change much of anything. Some people might
not need to buffer as much data, but otherwise, the end result is the same
- they end up sending a GOAWAY. It's just a question of "go away because I
realized you're sending me more headers than I want to deal with" (maybe we
can add a new error code for this if it makes people feel better), or "go
away because you sent me something I explicitly said I didn't want to
accept" (PROTOCOL_ERROR).

--
Peace,
  -Nick

On Mon, Jul 7, 2014 at 12:50 AM, Greg Wilkins <[email-masked]> wrote:

> The HTTP2 protocol has a requirement to be able to transport large
> headers, that exceed the payload size of a single frame at the current 16KB
> maximum size.
>
>    The total length of a HEADERS+CONTINUATION* sequence is not known
>    until the last frame in the sequence is processed. A receiver that wishes
>    to reject streams headers larger than a specific limit may have to process
>    The size of header that an endpoint is prepared to receive is not
>    known in advance. The only way a sender can know if a header too large is
>    by attempting to send it and receiving an error in response. Error handling
>    of headers may be difficult for an endpoint to handle efficiently and can
>    The END_STREAM flag is not present on the CONTINUATION frame, thus it
>    is possible for a stream to send CONTINUATION frames after a HEADERS frame
> This proposal increases the length field in the frame header to 31 bits,
> to match the maximum flow control window size. However, implementations
> will not be able to use the full frame size without explicit consent from
> peers using newly defined SETTINGS or an optional WINDOW_UPDATE field.
> Frame Size Settings
>
> Consider the example of a server that has commenced sending a large
> content to a client. The server may initially send 4 x 16KB frames to
> consume the default stream flow control window, at which time it must wait
> for the client to send a WINDOW_UPDATE frame before continuing. When
> generating the WINDOW_UPDATE frame, the client may have knowledge of:
>
>    - The content-type header - so it knows if the content has high
>    priority in rendering the current page, or if the content is likely to
>    include references to other resources which may need to be multiplexed.
> A minimally compliant implementation MUST handle the SETTING_FRAME_SIZE
> and SETTINGS_HEADER_SIZE and ensure that no frame sent exceeds the
> applicable limit. However no implementation is required to send frames at
> or near these limits when set above the default 16KB.
>
>    - It is not yet clear if extensions will be a viable way to enhance
>    the http2 protocol. There are significant hurdle to overcome to deploy
>    extensions.
>    - It is asymmetric to support large headers with one mechanism and
>    large data with another.
>
>
>
>    - There are options to have variable length headers or optional
>    extended headers that will preserve the semantics of this proposal and keep
>    an 8 byte header for small frames. If the 2 byte cost is considered
>    prohibitive, then these alterations can be considered.
>
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 20:17:57 +0200

Hi Nicholas,

> spend time and energy on this?

With all due respect, I think you've only expressed personal preference on
that subject with no defendable argument.

For example, the 0.02% you're talking about are the ones that *require*
support for CONTINUATION in *all* implementations and which make all
these implementations prone to slowloris-like attacks (for example).

Also concerning the numbers, I too have posted elements (please check
the archives) indicating that video already represents 60% of the
internet traffic and is deemed to 78% by 2018. *All* video traffic
will more or less significantly be hurt by the limit to 3 pages per
data block. Some people have already reported that 16kB over WiFi
is small and that doubling that size almost doubles the performance.
I think that allowing this value to easily grow in agreement with
both sides will leave more room for better supporting various
transport media.

Best regards,
Willy

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 7 Jul 2014 14:23:55 -0400

Hi Greg et al,

>    many frames and hold the results in memory before it discovers the header
This is orthogonal to CONTINUATION (and could also be addressed by
pre-pending the encoding length within the HPACK encoding, as has been
suggested). The problem with doing so is that it precludes the possibility
of streaming HPACK.

>    result in the closure of the entire connection.
It's not a requirement that an endpoint close the connection after deciding
that headers on a particular stream are too large. It does require that the
endpoint be willing to incrementally process header block fragments as they
arrive, to avoid unbounded buffering. With that, it's fully possible for an
endpoint to tightly bound it's memory commitment in the face of large
headers on a stream, without tearing down the connection.

Realistically as a service operator, even if given
SETTINGS_HEADER_FRAME_SIZE as proposed here, I'll still want to maintain
two thresholds of "large headers": A server-side threshold which sends an
error and tears down the stream, and a larger (potentially much larger)
client-side threshold. I want this because I otherwise have no
visibility into the impact that my selection has upon clients who wish to
communicate with me.

> Yes, this is a concern, but is easily solved (and is why Chromium is
emitting smaller continuation frames to ensure good interop coverage within
h2-13).

 cheers,
-johnny

---

From: Michael Sweet <[email-masked]>
Date: Mon, 07 Jul 2014 14:37:36 -0400

Willy,

On Jul 7, 2014, at 2:17 PM, Willy Tarreau <[email-masked]> wrote:
> ...

I think the key part of the proposal is that the receiver just has to say "I can take DATA frames up to this size" and the sender can choose how to best provide the data up to the advertised limit.  No agreement needs to be made here, it is entirely opportunistic.

---

From: Nicholas Hurley <[email-masked]>
Date: Mon, 7 Jul 2014 11:48:04 -0700

> these implementations prone to slowloris-like attacks (for example).

There will always be attacks. We take one away, another will be created. We
can mitigate this one without changing the entire framing of the protocol.
For example, moving END_STREAM to CONTINUATION instead of HEADERS is a very
quick, easy indicator of whether or not the header block has more frames
coming. That makes it easier to decide whether or not to kill the
connection after the HEADERS frame. We could also make a change to indicate
the total size of the header block before sending anything, which also
mitigates the slowloris-style attack (at the expense of not being able to
stream HPACK - a tradeoff I am thinking more and more that I would be
willing to make).

I'm also still not convinced by the state commitment issue. It's entirely
possible for an endpoint to process and discard header data (thus
maintaining the shared state for the connection) without having to hold on
to the resulting headers. This means that endpoint is only keeping the
state that it has already committed to keeping (namely, the shared HPACK
state). So, it turns out, I was wrong, you don't have to send a GOAWAY, you
can actually keep your connection going. This also allows you to send back
an HTTP-level error (say 413) which will be easier for people to debug when
they're faced with a server/intermediary that doesn't want to handle the
headers they sent.

Also, the argument that CONTINUATION support is untested is also
unconvincing. I know I've tested my continuation support (both sending and
receiving), against a few different servers. While I'm sympathetic to the
argument that we should remove footguns/untested code, there are always
going to be bugs, and this change just trades one footgun (possibly
poorly-tested continuation support) for another (almost certainly
poorly-tested large frame support).

> internet traffic and is deemed to 78% by 2018.

Let's please not conflate the issue of large headers with video. They're
only tangentially related - one is a correctness issue (large headers), one
is a performance issue. Let's assume some theoretical world where I end up
supporting large frames (VERY theoretical, at this point). If I don't know
that I'm receiving video, I don't want to advertise large frame support,
because when used poorly, that will kill multiplexing and priority, which
are essential to the non-video case. I generally won't know a priori
whether a connection I'm making will be used for video or not, so I don't
want to offer that performance footgun on every connection I make. And
while video is already a large percentage of the traffic, I haven't seen
any numbers so far that convince me it's a large percentage of the
connections. I'll say this again - I think jumbo/large/huge/whatever DATA
frames are a perfect candidate for an extension.
--
Peace
  -Nick

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 21:10:34 +0200

On Mon, Jul 07, 2014 at 02:37:36PM -0400, Michael Sweet wrote:
> I think the key part of the proposal is that the receiver just has to say "I
> can take DATA frames up to this size" and the sender can choose how to best
> provide the data up to the advertised limit.  No agreement needs to be made
> here, it is entirely opportunistic.

Absolutely. But that's what I call an agreement. It's not a negociation,
it's exactly like the TCP MSS. One side indicates its capabilities and the
other one feels free to use them if needed.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 21:30:12 +0200

On Mon, Jul 07, 2014 at 11:48:04AM -0700, Nicholas Hurley wrote:
> connection after the HEADERS frame.

Here it's irrelevant to this kind of attacks. Currently nothing prevents
a user-agent from sending headers one byte at a time using CONT frames.
Someone suggested that even 0 bytes might work as well. Quite an easy way
to consume all of a servers' resources with a very low bandwidth. It even
amplifies what we used to require over TCP because here you can even open
many streams inside a single TCP connection using a single TCP segment.

The issue is that due to the way the protocol is designed right now, there
is an easy incentive for not sending headers in one block, and poor designs
will probably do so, requiring servers to always support CONTINUATIONS and
be prone to such weaknesses. Not to mention the burden of dealing with them.

> We could also make a change to indicate

It could be better, but will not completely defeat such attacks, as you can
easily advertise 16kB of headers and send 16383 blocks of 1 byte.

> they're faced with a server/intermediary that doesn't want to handle the

But you still need to consume these continuation frames.

> poorly-tested large frame support).

Yes, but with the difference that it's more in line with common practices
in various internet protocols. CONTINUATIONS is similar to IP fragments.
No internet-facing server correctly supports them anymore, they are
nightmare everywhere, you're torn between dropping them all or trying to
reassemble them all and suffer from this. So everyone handles them using
heuristics or hacks. Filling packets with protocol-defined sizes is done
everywhere (IP, TCP, etc) and tends to be well done.

> Let's please not conflate the issue of large headers with video.

The draft is taking about "large frame format" and I thought you were
responding to that *in general*.

> They're
> want to offer that performance footgun on every connection I make.

I absolutely agree and that's exactly the purpose here : when you don't
know for a good reason you want to change the default, it remains unchanged.
However if some of your users start complaining that streaming a video from
their low-powered home NAS to their TV running your client is slow, you'll
probably be happy to propose them to experiment with a small change allowing
larger frames between the NAS and the TV so that the video is not jerky
anymore. At least that's much better than going back here and saying "guys,
we need to implement an extension, let's cross fingers for it not to break
intermediaries which are unaware of it, because it will change the framing".

Also, telling users that their video is slow because we want to ensure that
their TV gets a fair share of their home network's bandwidth to their NAS
will make them laugh at best, given that they're usually watching a single
video at once on a given screen.

> And
> connections.

I don't see the relation with connections here. For sure it must be much
less connections given they're much larger. What matters is end user's
experience. When a person spends 10 minutes watching a single-connection
video or 10 minutes reading news and loading 1000 objects, how are some
connections more important than the other ones ? What is important is
the number of times the user's experience is badly affected.

> I'll say this again - I think jumbo/large/huge/whatever DATA

And as I explained already, this will never ever happen just because
you have no way to ensure you can negociate a different framing without
breaking everything in the middle. TCP window scaling took something
like 10 or 15 years to take off because intermediaries which started
to inspect sequence numbers did not know that the window size they were
looking at was scaled and that it was normal that sequence numbers went
apparently out of windows. We'll get the exact same issue here with your
desktop's antivirus, your beloved web site's IPS, etc...

So framing is not a candidate for an extension.

Best regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 19:48:52 +0000

In message <CABkgnnW=+BXXQvf4dNqMG9UAqPZ4dXOqmMr8QNqY3=[email-masked]>

>well-enough supported, but this seems like feature creep to me.

Actually I suspect this will be a much more efficient way to
prioritize multiple competing streams than the complex priority
scheme currently proposed.

So feature creep ?  Well, yes.  But if it appears to make all the
priority stuff surplus to requirements, then it's a feature shaving
and much simpler to explain and implement.

>PUSH_PROMISE (we already have settings that govern PUSH_PROMISE).

Good catch!

>Any why 16384 rather than 16383?

Because computers have VM pages that are sized 2^n and many hardware
facilities work best in entire words.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 19:56:02 +0000

In message <[email-masked]>
, Nicholas Hurley writes:

>Anyone who cares to see my reasoning against

Please notice that this suggested change does not *mandate* use of
large frames, you are perfectly able and in your right to never
transmit or receive a single one of them.

All of the stuff I have found in the archives seems to be totally
blind to network technological developments such as fiber-to-the-home
and 4G, 5G and the recently started murmor about 6G mobile networks.

Limiting the protocol to 16K means it will be outdated almost before
it is ready to be used.

Our proposal is basically to future proof the protocol while retaining
as defaults the number based on the archives.

>spend time and energy on this?

Because the currently proposed solution for these 0.02% sucks ?

(NB: This "0.02%" number only covers HEADERS, our proposal also
allows large frames for DATA, which is a significantly larger market
than 0.02% of the traffic)

>there NEEDS to be a minimum on max frame size.

There is: 256 bytes, it's in the draft.  This number was chosen as the
smallest realistic number for highly tailored applications.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 20:02:35 +0000

In message <CAEn92To91LMwMzRubur5mWLfAc=[email-masked]>
, Johnny Graettinger writes:

>The problem with doing so is that it precludes the possibility
>of streaming HPACK.

While it may be an advantage (how big ?) for a client to be able to
"stream HPACK" it is a major inconvenience to everybody else in the
HTTP network topology to have no idea how much data is incoming.

Given that the CPU/MEM performance bottleneck is everywhere but the
client there needs to be really good arguments for shifting the
inconvenience of finding the total length from client to everybody
else.

Where are these arguments ?  I cannot find them.

It seems like "streaming HPACK" is an merely a neat idea and I have
certainly not seen any examples of clients that cannot function
without it ?

Poul-Henning

PS: In a previous email I outlined several strategies tiny clients could
use, I won't repeat them here.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 13:10:34 -0700

> many streams inside a single TCP connection using a single TCP segment.

That is true even with jumbo frames.
You send a large frame size, then send one byte and wait for a while, then
send another byte...
This pattern is a classic attack vector regardless of the framing
mechanism-- one must detect for it and handle it regardless.
Arguably, the jumbo-frame thing suffers more from this as, in order to get
CPU efficiencies, one allocates a large memory buffer for this instead of
resizing/partitioning, and thus commits a larger fraction of memory to the
attacker(s) performing the slowloris attack.

> easily advertise 16kB of headers and send 16383 blocks of 1 byte.

As mentioned above, an attacker can do the same thing with the jumbo frame.

> But you still need to consume these continuation frames.

Just as you need to consume the bytes from the jumbo frame.
The announced size of a frame doesn't change anything w.r.t. the DoS
surface.

> everywhere (IP, TCP, etc) and tends to be well done.

Given that we're going over an in-order, reliable transport, the analogy
isn't a strong one :)

> video at once on a given screen.

Note that the typical thought is that 'bigger is better', though we know it
to not be true w.r.t. latency.
This is part of the footgun. The good thing about the proposal as written
is that it takes two poor implementations to make for a poor experience,
which makes it likely that users will have adequate experience, at least
for the web case.
The downside of this is that we've added new and interesting failure cases
and latency requirements w.r.t. headers thanks to the buffering
requirements.

I can understand some of the frame-size arguments, though I've obviously
been less worried about them, and worry about adding more complexity to the
base framing (I prefer it be in the part that interprets the frames instead
of the framing itself).
The required-buffering of the whole headers stuff, however, still seems
like a bad idea. It increases the DoS surface area instead of decreasing it.
-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 20:31:30 +0000

In message <CAP+FsNed=[email-masked]>, Roberto Peon wri

>You send a large frame size, then send one byte and wait for a while, then

You can only do that with DATA frames.  The header-set has to fit a single
HEADERS frame.

Once the proxy/server has the complete header-set, it is in a much
better position to decide if this is part of a DoS attack or not and
consequently abort your DATA frame "slowlaris" attack.

The current CONTINUATION allows the DoS attack to progress before the
server has a chance to know anything it can use to decide on.

>resizing/partitioning,

Always allocating a worst-case sized buffer almost amounts to
efficiency malpractice in these days and times.

The "obvious" implementation is to read the 10 byte header first, then
find where it goes and read the rest.

A more "optimistic" implementation will read enough that the most
common "overhead" frames (at least WINDOW_UPDATE) fit in the first read,
and suffer the minor hit of copying a few tens of bytes if the
frame was not one of these.

>The announced size of a frame doesn't change anything w.r.t. the DoS

No, but it does help interop by offering clients useful information.

For instance the perviously mentioned CERN bulk-transfer application
could remove the brakes right up front, and go directly to very 
large frames.

It will also save our bacon if experience shows that the data sets that
caused the 16KB limit to be chosen were not representative of the
full universe of HTTP/2 candidates.

>to not be true w.r.t. latency.

Unless you only have a single stream, moving a very large file, in which
case it makes a heck of a difference to performance and latency.

Such applications are plentiful, for instance in pretty much any
electronic document journaling system or document retrival system.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 13:21:26 -0700

> HTTP network topology to have no idea how much data is incoming.

By data you're talking about the size of the compressed headers.
You're basing that argument about the amount of inconvenience on the
assertion that one achieves higher efficiency by allocating memory based on
what a possibly malicious client says should be the size.
I'm certain that you're right that it would be more efficient, until
attacked.

If we're following this argument to its logical conclusion, however, we
should also be communicating the *uncompressed* size, since we currently
have no idea how large we should be allocating buffers for that, and
arguably that is where the larger variance in amount of memory
consumed/buffer allocated would be.

> inconvenience of finding the total length from client to everybody
There is also the latency tradeoff, which for many usecases is the big
deal, and keeps being ignored.
Additionally, there is the DoS consideration, as mentioned before.
Since proxies are clients, and often even more constrained than servers,
requirements to buffer are potentially onerous, especially given that one
is not required to do this for HTTP today.

> certainly not seen any examples of clients that cannot function

The same can be said of jumbo frames, etc.

-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 20:16:48 +0000

In message <[email-masked]>
, Nicholas Hurley writes:

>Let's assume some theoretical world where I end up

You don't need to advertise large frame support: You can do it per
stream with the field in WINDOWS_UPDATE once you discover that this
particular stream carries video.

>because when used poorly, that will kill multiplexing and priority, 

"Deliver tools, not policies"

IP, TCP, TELNET and FTP could all be used poorly, but IETF members
were smart enough not to limit functionality just because some people
might not use them optimally.

Trying to circumscribe what people can do with your protocol (ie:
embedding politics in the protocol) has never been a success for
anybody:  People will just go and define a new protocol then (Guess
why we have started talking about HTTP/3 already?)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 20:47:36 +0000

In message <[email-masked]>, Roberto Peon wri

>what a possibly malicious client says should be the size.

Which is much better than the current drafts unbounded time/space
requirement for the server while the malicious CONTINUATION frames
trickle in.

>should also be communicating the *uncompressed* size,

I've actually thought about proposing that.

The reason I havn't is three-fold.

I suspect that most servers and proxies will not decompress more
than they absolutely need to.  If nobody cares about User-Agent,
why spend time de-huffman'ing it ?  Therefore they won't need to
know the size of the uncompressed headers.

Second it would impose a buffering requirement on client which through
"out-of-band" means know that they will not exceed the server limits.
Think of a lightbulb:  It knows that its request will need between
200 and 227 bytes, which is guaranteed to be acceptable, so there
is no (other) requirement to buffer it.

Third it is perfectly conceiveable that the client won't know until
it has composed and compressed the headers so the number wouldn't
be able until at the end.  To be useful for the server, it needs
to be up front (where space could be reserved before compression
is started - but complexity etc.)

If I'm wrong, it's a good candidate for one of the reserved bits later.

>deal, and keeps being ignored.

The only way client buffering of full HEADERS can introduce latency
is if the network bandwidth is approx 10 times higher than the
bandwidth at which the client can produce the HEADERS.

Which many usecases would that be ?

>is not required to do this for HTTP today.

You seem to forget that the proxies with the highest loads gets
the number for free from the client, and can just pass it on.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 7 Jul 2014 23:21:50 +0200

Hi Roberto,

> deal, and keeps being ignored.

It's not ignored, what is said is that in *some* circumstances it's not
the bigest deal and in such circumstances letting both sides agree on
how to optimally use resources is better than preventing them at the
protocol level. The example about the small NAS streaming to the TV is
a reasonably valid one. Such boxes have a very low CPU and are optimized
for sendfile() and will seriously suffer from copying data in small
chunks. There's no latency impact here, however the user sees a smooth
video.

I strongly believe that for the sake of latency keeping small frames by
default is the right thing to do and this proposal does so. It even allows
to reduce them even more if needed. Hey, if you connect to my home server,
you'll even see I negociate MSS to 512 to prevent outgoing HTTP traffic
from affecting my SSH sessions too much, so don't underestimate my concerns
with latency please :-)

> is not required to do this for HTTP today.

That's not true Roberto, proxies are required to honnor the Connection
header and to remove what is listed there, so in practice they cannot
start to forward even the smallest header because they don't know if
it will be tagged in a connection header later. And reverse-proxies
need to perform a minimum of security checks and cannot pass unverified
requests to servers (eg: with differing content-length headers). Also,
there are still a bunch of servers which send you a reset if you send
multi-packet requests (they're annoying to use with telnet), so that
practice in HTTP/1 is even further limited.

Cheers,
Willy

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 14:48:49 -0700

> from affecting my SSH sessions too much, so don't underestimate my concerns
I know-- I think I said somewhere that with the proposal, it would require
two parties before shots-in-the-foot started happening, which makes it at
least palatable, if not good tasting :)

Here is what I think about the various parts we've been discussing:

I could be convinced that stating the size of headers is useful. I don't
know if stating the uncompressed or compressed size is better. Regardless,
I don't think it should be a requirement. It is a nice hint, at best that
allows for slightly nicer buffer management under non-malicious
circumstances. It doesn't really change the DoS surface. Setting a limit on
the size of headers is something that I'd guess that we'd never use, since
it gives signals to attackers that allows them to optimize their attack
more than it gives us the ability to defend against the attackers. It might
have utility for embedded devices, though even in that case, I'd expect
that they'd do better to throw away keys/values that were uninteresting.
Meh.

I'd be perfectly happy to support having a setting that allows for up to
64k worth of data in a frame-- the framing mechanism would remain the same
and that'd take care of worries on my part about the base framing
mechanisms.
I see definite utility in this. It would still be possible for a poor
choice on the part of a gateway/proxy to mess up
priortization/multiplexing, but it'd be much less likely to happen, I'd
hope since a well implemented client plus a well implemented server would
prevent the proxy/gateway from at least some classes of poor implementation.

> practice in HTTP/1 is even further limited.

Those servers (which cannot accept requests of size > 1.4k :) ) sound
rather broken.
Proxies are required to examine connection, but, practically, gateways are
not required to honor the connection header.

The kind of reverse proxy matters here-- some will certainly do all kinds
of checks, DoS analysis, etc. Those are likely to buffer the whole request.

Others, which are worried more about latency, (potentially because they're
already behind a gateway which did the DoS analysis, etc) will not--
they'll simply direct the data to the correct processing node.

In the response path, gateways can have significantly more leeway than the
proposed change (requiring buffering) would allow-- today I can know, by
virtue of the fact that I'm running both the gateway and the server, that
the server isn't going to do anything that can't be checked on a key-value
basis (as opposed to a full-headers basis), and requiring buffering for
each gateway on that path induces more latency than needs to be there, and
worsens the tradeoff of adding a loadbalancer (which is typically a latency
for utilization improvement (and 2nd order effects of that)) tradeoff.

-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 21:58:13 +0000

In message <CAP+FsNdq4O0awp=B=5EPFh4y5XN=[email-masked]>, Roberto Peon wri
tes:

>each gateway on that path induces more latency than needs to be there, [...]

Can you please tell me, blow by blow, packet by packet, where that latency
comes from, because I can't find a place to fit it in ?

Just because you get your HEADERS in a single frame doesn't mean you cannot
process it byte by byte (or MTU by MTU) if you want to ?

The only real difference is that you get to know up front how much
will be coming your way.  I have a hard time seeing how that can
hurt your latency ?

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 15:04:43 -0700

Oy.

The concept is fairly simple.

If you need to know the entire size of the headers before one is allowed to
send any of it (as would be the case), then one must wait for all of the
headers data to arrive (and mutate it) before forwarding.
Thus, adding at a minimum: header-bytes/bandwidth seconds of latency per
gateway (not to mention the additional memory for the buffering).
I'd expect this to add a 10th of a ms per gateway or so, more on more
constrained links.

-=R

On Mon, Jul 7, 2014 at 2:58 PM, Poul-Henning Kamp <[email-masked]>
wrote:

> In message <CAP+FsNdq4O0awp=B=5EPFh4y5XN=
> [email-masked]>, Roberto Peon wri
> tes:
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 22:07:40 +0000

In message <CAP+FsNffcTXwbTK55fh3uGJ0n-MAy=[email-masked]>, Roberto Peon wri

>headers data to arrive (and mutate it) before forwarding.

And do you have any real-world examples where this would be the case ?

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:13:00 +1000

On 8 July 2014 08:04, Roberto Peon <[email-masked]> wrote:

> If you need to know the entire size of the headers before one is allowed
> to send any of it (as would be the case), then one must wait for all of the

Hey this is double jeopardy!   When I've previously pointed out the problem
of DoS deadlock by sending a HEADERs and never sending the promised
CONTINUATION, the response has been that no intermediary should every
forward a HEADERs until it has the entire header block.

So it can't be argued both ways - either we come up with something that
truly allows streaming headers (ie no mutating of shared state) OR we
accept that all headers have to be buffered.

> constrained links.

only for the 0.001 of traffic that has large headers.  Cry me a river!

cheers

---

From: Jason Greene <[email-masked]>
Date: Mon, 7 Jul 2014 17:13:54 -0500

On Jul 7, 2014, at 5:04 PM, Roberto Peon <[email-masked]> wrote:

> If you need to know the entire size of the headers before one is allowed to send any of it (as would be the case), then one must wait for all of the headers data to arrive (and mutate it) before forwarding.
> Thus, adding at a minimum: header-bytes/bandwidth seconds of latency per gateway (not to mention the additional memory for the buffering).
> I'd expect this to add a 10th of a ms per gateway or so, more on more constrained links.
> 

If it’s HTTP/2 you know the length by just quickly reading the inbound header. Or did you mean relaying HTTP/1.1 -> HTTP/2?

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 8 Jul 2014 00:22:17 +0200

> I'd expect this to add a 10th of a ms per gateway or so, more on more

You're generally much below, and I think you'll measure much better on
your own gateways (cause I know you optimize for this). A long time ago
I measured haproxy's parsing of headers at 1.2 microsecond on a pentium-M
1.7 GHz (my laptop by then) for an average request coming from my browser.
The rest is connection establishment (you don't care in the response) and
NIC's IRQ latency + switch to NIC serialization + TCP stack. On an 1 Gbps
port, serializing a 300 bytes request over the wire takes 4 microseconds.
NIC's IRQ delivery is generally the worst, between 40 and 100 microseconds
in general.

But anyway I can easily understand that for your specific use case, when
you write both the server and the gateway, you can consider both of them
as a single component and have each one off-load part of the job to save
the other one's resources. So I'm not shocked by seeing that you don't
process responses for example.

Thanks for the details,
Willy

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:23:54 +1000

> suggested). The problem with doing so is that it precludes the possibility

Streaming HPACK is already not possible as it creates DoS vectors.
Either we fix that problem - ie make header blocks able to be fragmented
and interleaved,  or we have to face up to the fact that we already have
jumbo header frames.

> headers on a stream, without tearing down the connection.

Indeed - but a lot of work for nothing. Best just to tell the peer not to
send them in the first place.

> emitting smaller continuation frames to ensure good interop coverage within

Which is precisely the reason I have moved from the
I-can-live-with-CONTINUATIONS-because-I-can-ignore them camp, back to the
I-can-not-live-with-CONTINUATIONS camp.      Several proposals have been
made to make them ignorable for small headers, but as they have not been
accepted then it looks like we all must support continuations and
implementations that do not will be hunted down and excluded from
interoperability, even if they never ever need large headers.

Mark - can you open an issue for this one-   CONTINUATIONS looks like it is
only needed for large headers, but it is actually a non optional part of
the specification.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:31:25 +1000

> that implementations will provide and respect it.

Martin,

well we could make it compulsory if that makes the proposal more likely to
be accepted :) We could mandate that all implementation MUST make random
fluctuations to the max frame size to hunt down and shame those
implementers who stupidly think that they need not implement a feature that
they do not need. [ Note that was sarcasm for those who don't get it ]

I think the WINDOW_UPDATE is well motivated to support the known use-cases
where multiplexing is not needed momentarily.     However the proposal
stands on without it and it could be removed if the WG explicitly
acknowledges that they will not support the issue that it is trying to
address (or comes up with an alternative).

But I'd ask the WG to mull on the proposal in total for at a little bit
before we abandon parts of it.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:36:49 +1000

On 8 July 2014 04:48, Nicholas Hurley <[email-masked]> wrote:

> Let's please not conflate the issue of large headers with video

+1

I actually think that if you asked those that server video, they are not
likely to want huge frame sizes.  There is little point sending video much
faster than it can be watched, as to do so means that you need lots of
buffering in the client and when they decide they've watched enough of the
very cute kittens and press stop, you've sent a lot of bytes that were not
consumed.

OK I know it is actually impossible to stop watching a cute kitten video,
but I'm sure there are use-cases to which my argument applies.

Video is a use-case where frame size probably needs to be optimised, but
not necessarily up.

cheers

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 7 Jul 2014 18:39:36 -0400

> and interleaved,  or we have to face up to the fact that we already have

It's not possible in the general case for a proxy with no application
knowledge of what it's proxying and/or reason to distrust it's peers.
That's not the same as saying that streaming HPACK isn't possible anywhere,
ever.

 It's not a requirement that an endpoint close the connection after
>> deciding that headers on a particular stream are too large. It does require
>> that the endpoint be willing to incrementally process header block
>> fragments as they arrive, to avoid unbounded buffering. With that, it's
>> fully possible for an endpoint to tightly bound it's memory commitment in
>> the face of large headers on a stream, without tearing down the connection.
>>
>

I believe this isn't true, but am restricting that conversation to the
#541: CONTINUATION thread. *tl;dr*: You're gonna have to do it anyway.

> Mark - can you open an issue for this one-   CONTINUATIONS looks like it
> is only needed for large headers, but it is actually a non optional part of
> the specification.
>
> cheers
>
>
>
> --

---

From: Martin Thomson <[email-masked]>
Date: Mon, 7 Jul 2014 15:40:34 -0700

On 7 July 2014 15:31, Greg Wilkins <[email-masked]> wrote:
> fluctuations to the max frame size to hunt down and shame those implementers
> who stupidly think that they need not implement a feature that they do not
> need. [ Note that was sarcasm for those who don't get it ]
>

I don't think that I was clear enough.  I really don't like that part,
compulsory or not.  I think that it's an unvarnished attempted to cram
per-stream settings into the protocol.  It's unnecessary and very
poorly supported.

A simpler and better approach for those connections that need generous
frame sizes occasionally would be to advertise a large setting.  As
PHK is fond of telling us: you don't have to use all of it.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 15:43:35 -0700

> and interleaved,  or we have to face up to the fact that we already have
Not all use-cases have potentially malicious users, and jumbo frames do not
solve the blocking problem.
Making frames after the first one non-state-modifying or
non-state-referencing, however, would solve that problem.

> send them in the first place.

And what if you're forwarding to another multiplexing proxy, and only then
to a server?
Which limit applies to which request?
It gets complicated and unuseful, and one needs to be able to reject
larger-than-wanted headers anyway.

-=R

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:47:35 +1000

On 8 July 2014 03:53, Nicholas Hurley <[email-masked]> wrote:

> So now we're resurrecting jumbo frames.
>

Not resurrecting.  CONTINUATIONs are jumbo frames, so we already have
them.  We are just making them sane

> Again. I've already said "no" to this, and apparently I have to say it
> again. No jumbo frames, no large frames. Anyone who cares to see my
> reasoning against jumbo/large/huge/whatever frames is welcome to go back
> and read the archives.
>

I care, so I've gone back to see your arguments:

On 26 June 2014 04:48, Nicholas Hurley <[email-masked]> wrote:

> Exactly. Jumbo frames seem like prime candidates for an extension,
> especially since no one can seem to come together on a single plan for what
> they would look like.
>

We've had a significant group get together and have decided what they
should look like.  please consider on merit.

> And given that we've already removed things from the spec that had some
> actual implementation experience (or at least broad implementer interest -
> I'm thinking ALTSVC and BLOCKED), I see no reason to add something to the
> spec just before WGLC.
>

Why go to LC, when those calls will be many saying they can't live with
continuations?

> If the procedural concerns aren't enough (they don't seem to have been in
> the past, so I'm not sure they will be now),
>

they are not enough.

> let's talk about actual experience. We have running code, right now, from
> a good number of implementers, that works with the regular-sized frames as
> currently in the spec. The current frame layout is simple, easy to parse,
> and easy to make decisions about. With all the perceived "complexity" that
> people have been complaining about, I'm honestly a little surprised that
> people are asking to add even more complexity (some of whom seem to be the
> same people complaining earlier!)
>

I believe the proposal is seen as simpler by most.  It certainly has a less
complex state machine.  It just standardises limits that many
implementations will already have. The complexity of changing a 14 bit
integer to 31 bits in the header can hardly be prohibitive.

I'm not convinced by the arguments around large file downloads - those are
> the exception rather than the rule. Optimize for the common case. In HTTP,
> that means viable multiplexing and priority, both of which are much more
> effective with frame size limited to 16k like we have it now.
>

In which case why are we getting ourselves in knots over the 0.001% of
cases with large headers.   If large headers get support in the standard
spec, then surely large payloads should also.

Besides, many of the issues the proposal addresses are for tuning
multiplexing for future networks - this is the common case.

>
> This is a HUGE change to the spec in order to make people feel better
> about cases that make up 0.02% of the streams out there. Why are we
> continuing to spend time and energy on this?
>

Hold a sec - you can't say it is a "HUGE change" and then later say it
doesn't "change much of anything".   It is a moderate change.  Increase the
length field to 31 bits and use settings rather than capacity to limit
frame size.    Not rocket science.

> CONTINUATION -- I'd be fine with changing in the existing spec. Put the

It does not help unless you allow interleaving.

> people two directions (too low and too high) to shoot themselves in the

+1, the proposal says 256, but happy to discuss better minimums.  I think
1024 would be fine.

> accept" (PROTOCOL_ERROR).

As a server, I'd much rather the client did the buffering and then didn't
send a too large header.   Having the server buffer up all the headers for
the clients and then throw them away if they are too large is a poor design.

regards

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 22:52:27 +0000

In message <[email-masked]>, Greg Wilkins wri
tes:

>faster than it can be watched, [...]

Depending on the quality.

Have you talked to any of the people doing 4K streaming ?

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 22:54:36 +0000

In message <CAEn92ToAwPCLgwPRg+p7bKqJ4dv4qHka5neW3pZBdDP=[email-masked]>, Johnny Graetting

>That's not the same as saying that streaming HPACK isn't possible anywhere,

DoS attacks are almost universally available and writing a protocol
not obviously vulnerable to them is much more important than to
cater for whatever the use-case you keep talking about, but which
I cannot get you to give a concrete example of might be...

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 22:57:33 +0000

In message <[email-masked]>, Martin Thomson w

>PHK is fond of telling us: you don't have to use all of it.

But if you advertise a larger max with SETTINGS it applies to all streams.

What if you only want to pamper that one video-stream or conversely,
only want to handicap that background upload ?

I think per-stream settings are missing in the current draft and most
of the priority stuff seems to be a poor substitute for it.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 08:56:50 +1000

On 8 July 2014 08:43, Roberto Peon <[email-masked]> wrote:

> Not all use-cases have potentially malicious users, and jumbo frames do
> not solve the blocking problem.
>

Not claiming it does.  Just saying that streaming is not generally
supported and thus is not a string argument against the proposal.

> non-state-referencing, however, would solve that problem.

To this I make the response that has been made to jumbo frames - there
appears to be lots of discussion about how/if this is possible and it is
unclear if there is a single proposal.  A pull request would be helpful to
consider if this is a real proposal.

> larger-than-wanted headers anyway.
This is a straw man argument as it pretends that these limits/problems
don't already exist in http/1 and the current draft.   There are limits all
along the path and headers can and do get rejected because of size.

All we are proposing is that a hop declares what it is prepared to accept.
This makes no representation what size up-stream or down-stream entities
will accept and is not an end to end guarantee.

Not having the setting and just a 14 bit length does not guarantee that a
proxy will accept the header and it can still send a 431.

The limit on frame size is not to give an end to end guarantee.   It is
there to configure multiplexing efficiency for the next hop. That's all, no
more no less. And that is a very valuable thing to do.

cheers

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 23:02:29 +0000

In message <[email-masked]>, Roberto Peon wri
tes:

>And what if you're forwarding to another multiplexing proxy, and only then to a server?

Simple: You always respect the one your peer tells you.

Your peer may be a proxy that needs your elephantine Kerberos Cookie
but does not forward it to the server.

Or it may have a better compression state with the server and be
able to squeeze your header-set through.

It might even be in cohorts with the server (ie: CDN) and strip out
most of the headers that your browser needlessly spits out, before
forwarding a much smaller request to the distant server over a thin
slow pipe.

You almost invariably end up worse by trying to second-guess the proxy.

>It gets complicated and unuseful [...]

No, it's simple, and I just showed you three valuable use-cases.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 7 Jul 2014 16:04:08 -0700

> But if you advertise a larger max with SETTINGS it applies to all streams.

Correct.

> only want to handicap that background upload ?

Flow control perhaps?

> I think per-stream settings are missing in the current draft

Whether you agree or not, that is entirely intentional.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 16:06:42 -0700

On Mon, Jul 7, 2014 at 4:02 PM, Poul-Henning Kamp <[email-masked]>
wrote:

> In message <
> [email-masked]>,
> Roberto Peon wri
> tes:
>

You just showed me three use-cases where the other request which are
multiplexed along that connection on either end will suffer failure because
the limit was too small, or the server/proxy accepted the work because the
end-users behind that connection need different limits.

Lets make it concrete.
Client A,is speaking to a proxy B, to servers C, D.
Server C wants a max header limit of 4k.
Server D wants a max header limit of 8k.

What does proxy B do?

-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 23:09:49 +0000

In message <CABkgnnXvvHRBCH3F3xV=[email-masked]>, Martin Thomson w

>> only want to handicap that background upload ?

Using the per-stream window to restrict frame size costs you one
RTT per frame.

It would certainly work for penalizing a stream but not for pampering
one with more throughput.

>Whether you agree or not, that is entirely intentional.

Intentional is neither a synomym for smart nor correct :-)

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 16:10:55 -0700

> > only want to handicap that background upload ?

And/or priorities.. that is their purpose.

> Whether you agree or not, that is entirely intentional.
+1
I originally had an idea to have stream groups, but found it unworkable.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Jul 2014 23:16:51 +0000

In message <[email-masked]>, Roberto Peon wri

>Server D wants a max header limit of 8k.

What is proxy B's job? 

If proxy B is a corporate SOX-compliance proxy that needs your
kerberos ticket to let you through, it will tell you that it
takes 64K frames and send you 413 if the headerset is bigger
than what C or D (depending on Host:) will accept, after having
stripped the Kerberos ticket out.

If proxy B is a CDN with two servers behind RFC1149 connectivity,
it will announce something big enough for what you might send
and then it will file away at the headers, in contract with
server C and D, until they are small enough to tied to the legs.

If proxy B is a load-balancer in front of C and D, it will
advertise the max, ie 8K, and if you send an 8K header
destined for server C, it will either send you a 413 or
in contract with server C rewrite the request so server
C gets to emit the 413.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 7 Jul 2014 16:22:42 -0700

On Mon, Jul 7, 2014 at 4:16 PM, Poul-Henning Kamp <[email-masked]>
wrote:

> In message <
> [email-masked]>,
> Roberto Peon wri
> tes:
>
> >Lets make it concrete.

Sure.

> server C and D, until they are small enough to tied to the legs.

And if it can't, then it will 413 the request for lack of any other option.

> in contract with server C rewrite the request so server

Agreed.
I'll note that we're still needing to send 413s from
proxies/loadbalancers/gateways in many cases.

-=R

---

From: <[email-masked]>
Date: Mon, 7 Jul 2014 23:24:35 +0000

>I originally had an idea to have stream groups, but found it unworkable.

Hold on, we do already have pseudo per-stream settings. Even if you don't call it a per-stream setting, each stream has it's own flow control window that's set at stream begin by SETTINGS_INITIAL_WINDOW_SIZE.

Perhaps it would be better to say SETTINGS_INITIAL_MAX_FRAME_SIZE, but I hardly think keeping a second integer for each stream is "complex". Yes, it adds more state, but it is optional.

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 09:58:10 +1000

> Server D wants a max header limit of 8k.

Again this is a strawman.   This problem exists today for http/1 and h2-13.
Having a declared limit in no way changes the fact that there are limits.

Proxy B can do whatever it likes. It can have a 4K limit and all is fine.
It can have an 8k limit and then  when it receives a >4K <8KB header that
is destined for Server C, it responds with 413.  Or it forwards it anyway
and server C 413s or go-away - just as it does today.

The proposal only makes this problem more explicit. It does not make it
worse, but it may make it easier to see and avoid in the first place.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 10:02:22 +1000

> > only want to handicap that background upload ?

I indeed think that the dynamic max frame size is part of the flow control
calculations, hence that is why I proposed it as part of the WINDOW_UPDATE
and not as SETTINGS.

You can see that the example given that the calculation to adjust the max
frame size is done as part of the calculation for window size.

I think that allowing a dynamic stream max frame size is just another
useful lever that some flow control algorithms will like to pull.  Others
will not, but then others also will not adjust the 64KB window either.
The whole point of the current flow control design is to allow flow control
to develop over time with experience.  Having frame size as part of that
will be valuable to explore the space of possible solutions.

cheers

---

From: Jason Greene <[email-masked]>
Date: Mon, 7 Jul 2014 19:04:35 -0500

On Jul 7, 2014, at 6:22 PM, Roberto Peon <[email-masked]> wrote:

> 
> 
> 
> On Mon, Jul 7, 2014 at 4:16 PM, Poul-Henning Kamp <[email-masked]> wrote:
> I'll note that we're still needing to send 413s from proxies/loadbalancers/gateways in many cases.
> 

The point is not to eliminate 413s though. The point is to mitigate HOL blocking and significantly reduce connection drops between hops. Although you also get a nice benefit of reducing the total processing cost of participants.

---

From: Mike Belshe <[email-masked]>
Date: Mon, 7 Jul 2014 17:07:55 -0700

> of multiple streams over the specific connection.

Greg -

Overall, great writeup.  Thank you.

Clarifying question:  are you bundling these settings parameters because
you think we need them?  Or simply as a compromise?  If you had to choose
between a larger frame size and no settings change vs the current frame
size, which would you find simpler?

Thanks,
Mike

---

From: Martin Thomson <[email-masked]>
Date: Mon, 7 Jul 2014 17:23:20 -0700

> frame size is done as part of the calculation for window size.

It's only that way because you made it that way.  The common factor is
that they deal with incoming bytes.  That's not a particularly strong
argument.

> I think that allowing a dynamic stream max frame size is just another useful
> lever that some flow control algorithms will like to pull.

"I think" doesn't really go far enough to justify what is a pretty
fundamental change to the structure of the protocol (far more so, in
my opinion than the relatively arbitrary change to frame size).

We could build all sorts of knobs, levers and dials.  The key to
success doesn't involve finding all those that can be justified and
add them, it involves finding all those that can be done without and
removing them.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jul 2014 10:58:40 +1000

On 7 Jul 2014, at 9:44 pm, Greg Wilkins <[email-masked]> wrote:

> I agree that the issues need to be carefully identified.   I think 549,550 and 551 are all good issues to create (although this proposal does not directly address 550).   But I think there should also be issues for 
> 	• a fixed frame size does not allow tuning multiplexing performance based on current/future experience.
> 	• the 16KB frame size is not well tuned for frequent payload sizes
> 	• a fixed frame size cannot be adjusted for specific streams in specific situations that may not required multiplexing efficiency.
> Or at least one issue that the fixed max frame size cannot be tuned for any reason

That's a design decision that was made a long time ago. To reconsider it, I need more than vague concerns that amount to "I don't like it"; I need concrete problems that it causes.

> One note of caution with decomposing to individual issues, is that no single issue may be sufficiently important to rock the boat at this late state, but a proposal that resolves many such issues might well be worth the disruption.

... and that's why I've asked for the WG to take this proposal seriously -- as a potential path to resolving a number of related issues. Strictly speaking, though, it does a lot more than is necessary to get there. 

On 8 Jul 2014, at 8:23 am, Greg Wilkins <[email-masked]> wrote:

> Mark - can you open an issue for this one-   CONTINUATIONS looks like it is only needed for large headers, but it is actually a non optional part of the specification.

That's at most an editorial issue, and probably an invalid one; no where does the spec say you can ignore CONTINUATION, or that it's optional.

Cheers,

---

From: Michael Sweet <[email-masked]>
Date: Mon, 07 Jul 2014 21:46:55 -0400

Greg,

On Jul 7, 2014, at 6:36 PM, Greg Wilkins <[email-masked]> wrote:
> I actually think that if you asked those that server video, they are not likely to want huge frame sizes.  There is little point sending video much faster than it can be watched, as to do so means that you need lots of buffering in the client and when they decide they've watched enough of the very cute kittens and press stop, you've sent a lot of bytes that were not consumed.
> 
> OK I know it is actually impossible to stop watching a cute kitten video, but I'm sure there are use-cases to which my argument applies.
> 
> Video is a use-case where frame size probably needs to be optimised, but not necessarily up.

Frame size optimization means balancing (perceived) latency with bandwidth/processing efficiency.  Let's call this the LRRH ("Little Red Riding Hood") principle - not too big, not too small, "just right".

For printing today the optimal frame size seems to be 64k.  I'm guessing that video will be similar, if only because the data volume is similar and interruptions in the data stream can cause stuttering or paused output.  A 1080p MPEG-4 movie file on iTunes averages about 28k per video/audio frame (24 frames per second) - you'll need two 16383-octet DATA frames to hold that, while a single 64k DATA frame can hold two video/audio frames.

So I actually think that many bulk data "pipes" will desire to use a larger frame size than 16383, but that size will reflect a balance of what is "just right" for the application - not infinite, but a number that provides perceived/visible improvement without adversely affecting things for any of the involved parties, otherwise you can kiss any improvement goodbye.

---

From: Michael Sweet <[email-masked]>
Date: Mon, 07 Jul 2014 22:06:18 -0400

Mark,

> That's a design decision that was made a long time ago. To reconsider it, I need more than vague concerns that amount to "I don't like it"; I need concrete problems that it causes.

For printing 16k frames/chunks cause a 2x performance drop.  And for video streaming you'll likely see similar issues (and stuttering) due to the volume of data involved. (see my other post on the subject)

---

From: Michael Sweet <[email-masked]>
Date: Mon, 07 Jul 2014 22:08:46 -0400

(please ignore this very tired man's mixup on children's stories - I did mean Goldilocks, of course...)

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 13:14:28 +1000

Mike,

I think the settings are needed with the ability to send larger frame
sizes.   If I understand the feedback from SPDY correctly, the larger frame
sizes there were some endpoints that were lazy with their frame sizes and
sent overly large ones that hurt multiplexing.

The intention of the proposal is to allow large frame sizes IF NEEDED, but
to use the settings to constrain implementations to the 16KB that has been
selected as a reasonable one-size-fits-all for todays traffic.   The
settings can then be adjusted without needing to rev the spec or deploy
extensions as experience is gained, traffic changes, networks change
etc.     It may well be that they are adjusted down initially as much as
they are adjusted up.

So I don't see it as a compromise - it is simply moving the limit that must
exist from being a fixed capability of the framing layer to be an
explicitly managed parameter of the protocol.

cheers

On 8 July 2014 10:07, Mike Belshe <[email-masked]> wrote:

>> This proposal has been prepared as it is possible to meet the
>> requirements of CONTINUATIONS without the complications and criticisms
>> above.
>>
>> To handle the issue of efficiently sending large data when an end point
>> is prepared to risk multiplexing efficiency, this proposal allows a Max
>> Frame Size to be applied to a specific stream as an optional field in a
>> WINDOW_UPDATE frame.
>>
>>    - How many other requests are pending - so it knows if new
>>    multiplexed stream could soon be opened.
>>    - An approximate rough measure of the network latency and throughput
>>    - This can be derived from the timing of the receipt of the first few data
>>    No change. It can just not adjust the frame size, either because it
>>    is too hard to consider or that there are too many other streams, or that
>>    the content is video that needs to be received slowly. In any of these
>>    cases the max frame size can be left unchanged and the protocol continues
>>    as it currently does.
>>    -
>>
>> There is no requirement for an implementation to send or to handle the
>> Max Frame Size in a WINDOW_UPDATE and it is allowable for it to be ignored
>> if received.
>> Anticipated Feedback
>>

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 13:20:03 +1000

On 8 July 2014 10:58, Mark Nottingham <[email-masked]> wrote:

> That's at most an editorial issue, and probably an invalid one; no where
> does the spec say you can ignore CONTINUATION, or that it's optional.

That is the issue!

Implementations are required to implement a modal feature that is only used
by 0.001% of traffic, with a switch between modes at an indeterminate size
and many implementations will never ever ever see any traffic that uses
this mode.

This is exactly like header field continuations in http/1 - a required
seldom implemented (correctly) feature used by infinitesimal users.

At the very least we should acknowledge that this is a problem, even if in
the end we accept it so we can handle big headers.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 13:23:55 +1000

On 8 July 2014 10:58, Mark Nottingham <[email-masked]> wrote:

> That's a design decision that was made a long time ago. To reconsider it,
> I need more than vague concerns that amount to "I don't like it"; I need
> concrete problems that it causes.

insert Michael Sweets use-case here.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 13:39:11 +1000

I've pushed some fixes to a few editorial issues in the proposal identified
by Martin

https://github.com/gregw/http2-spec/commit/b55765fc173823da1e515ec653604d9cea6ce820

cheers

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jul 2014 13:46:20 +1000

431, not 413.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jul 2014 13:48:41 +1000

On 8 Jul 2014, at 8:43 am, Roberto Peon <[email-masked]> wrote:

> It gets complicated and unuseful, and one needs to be able to reject larger-than-wanted headers anyway.

It's effectively hop-by-hop; you're moving the application of the next hop's policy regarding header sizes to the sender (while still likely enforcing it at the recipient, albeit with messier consequences).

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jul 2014 14:10:04 +1000

Michael,

> For printing 16k frames/chunks cause a 2x performance drop.  And for video streaming you'll likely see similar issues (and stuttering) due to the volume of data involved. (see my other post on the…

Out of curiosity - are your use cases *actually* using HTTP/2, or are you assuming some mapping of IPP to HTTP/2?

Thanks,

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 8 Jul 2014 14:12:40 +1000

All,

Here is my summary of objections to this proposal so far.

>From my biased point of view, the following objections have been raised but
adequately responded to:

   - Too late in the process
      - It is too late when the chair tells us it is too late
   - The lack of ability to tune frame size has not yet definitely been
   identified/accepted as an issue.
      - real use-cases have been provided where a 16KB is far from optimal.
      - I don't like jumbo frames
      - I don't like continuation - deadlock!
      - Could achieve the same with a prepended size to a
   HEADERS+CONTINUATION* sequence
      - Worst of both worlds
      - Doesn't solve all DOS attacks
      - never claimed it did
   - It is too complex
      - Perhaps WINDOW_UPDATE, but the core change is very simple.

But I do think we need to discuss these ones some more:

   - The WINDOW_UPDATE change is adding a feature
   - The WINDOW_UPDATE change is a per stream setting - thus it is bad.
      - The mechanism is intended to be part of the flow control
      algorithm.   It is the intention of http2 to not specify a flow control
      algorithm, but instead define the bounds of flow control and to then let
      implementations develop better algorithms over time, with experience and
      potentially for special purposes.   Adding frame size as a variable that
      flow control algorithms can adjust will allow for more algorithms to be
      explored without requirements to change the specification or deploy an
      extensions.   However, this part of the proposal is indeed separable from
      the rest of it.

   - It assumes 'bigger is better' and will make for a poor QoS.
      - It is definitely not the intention of this proposal to just allow
      bigger frames, rather to move the frame size limit from a fixed
capacity to
      a configurable parameter.  In some circumstances it may well be that the
      frame size is configured smaller rather than larger.   I think
it is really
      important that we explore this issue an make sure that the proposal does
      not allow inadvertent nor deliberate degradation of QoS and
while i believe
      this is already the case, we need to ensure that it is understood.

   - It prevents streaming headers
      - yes it does (no such thing as a free lunch!) However, other
      limitations of hpack have already severely limited the practicality of
      streaming headers.  In fact I've not yet seen a good description of a use
      case where a header can be safely streamed without risking HOL
blocking.
      I believe that the benefits of this proposal outweighs this particular
      limitation.  If streaming headers really are a hard requirement, then we
      should look to HPACK to make changes, not the framing layer.

Have I missed any?

cheers

On 8 July 2014 13:39, Greg Wilkins <[email-masked]> wrote:

> I've pushed some fixes to a few editorial issues in the proposal
> identified by Martin
>
>
>>> That's a design decision that was made a long time ago. To reconsider
>>> it, I need more than vague concerns that amount to "I don't like it"; I
>>> need concrete problems that it causes.
>>
>>

---

From: Michael Sweet <[email-masked]>
Date: Tue, 08 Jul 2014 08:15:52 -0400

Mark,

> Out of curiosity - are your use cases *actually* using HTTP/2, or are you assuming some mapping of IPP to HTTP/2?

I'm applying our experience with HTTP/1.1 chunk sizes to HTTP/2; as I mentioned in my original (more detailed) response yesterday, there are no HTTP/2-based IPP printers (yet), but that is one of the projects I am working on right now... Since IPP is just another POST-based RPC scheme (using the IPP binary message encoding vs. XML or JSON) the only real change here is to using HTTP/2 in place of HTTP/1.1 as the transport.

The original message was in response to claims that smaller frame sizes will not adversely affect performance, but I provided experience with running code that we saw issues in HTTP/1.1 with smaller chunks and that a modest increase in chunk size provided  a dramatic performance improvement for all printers.  There is no reason to expect any different experience in HTTP/2, as the HTTP/2 frame overhead (8 bytes currently) is generally the same as the HTTP/1.1 chunking overhead ("3fff\r\n" + "\r\n" = 8 bytes)  with similar CPU processing requirements.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 8 Jul 2014 21:45:06 +0900

On Tue, Jul 8, 2014 at 9:15 PM, Michael Sweet <[email-masked]> wrote:

> >> For printing 16k frames/chunks cause a 2x performance drop.  And for
> video streaming you'll likely see similar issues (and stuttering) due to
> the volume of data involved. (see my other post on the subject)
> >
> > Out of curiosity - are your use cases *actually* using HTTP/2, or are
> you assuming some mapping of IPP to HTTP/2?
>
> I'm applying our experience with HTTP/1.1 chunk sizes to HTTP/2; as I
> mentioned in my original (more detailed) response yesterday, there are no
> HTTP/2-based IPP printers (yet), but that is one of the projects I am
> working on right now... Since IPP is just another POST-based RPC scheme
> (using the IPP binary message encoding vs. XML or JSON) the only real
> change here is to using HTTP/2 in place of HTTP/1.1 as the transport.
>
> The original message was in response to claims that smaller frame sizes
> will not adversely affect performance, but I provided experience with
> running code that we saw issues in HTTP/1.1 with smaller chunks and that a
> modest increase in chunk size provided  a dramatic performance improvement
> for all printers.  There is no reason to expect any different experience in
> HTTP/2, as the HTTP/2 frame overhead (8 bytes currently) is generally the
> same as the HTTP/1.1 chunking overhead ("3fff\r\n" + "\r\n" = 8 bytes)
>  with similar CPU processing requirements.
>
>
​I did some tests 16383 vs 65535 DATA payload performance in nghttp2
implementation on my desktop PC.
The result is 64K version performed better by 1%, which I think is
negligible.
Maybe embedded devices like printers are far more sensitive.  For server
side software and desktop client, payload size is not so much concern, no?
 Are there anyone who actually measured these settings on their HTTP/2
proxy implementations?

Best regards,
Tatsuhiro Tsujikawa

​

---

From: Michael Sweet <[email-masked]>
Date: Tue, 08 Jul 2014 08:52:39 -0400

On Jul 8, 2014, at 8:45 AM, Tatsuhiro Tsujikawa <[email-masked]> wrote:
> ...
> ​I did some tests 16383 vs 65535 DATA payload performance in nghttp2 implementation on my desktop PC.
> The result is 64K version performed better by 1%, which I think is negligible.
> Maybe embedded devices like printers are far more sensitive.  For server side software and desktop client, payload size is not so much concern, no?  Are there anyone who actually measured these settings on their HTTP/2 proxy implementations?

For a printer, every chunk/frame of data gets passed off to the print engine for processing.  If the chunk doesn't contain enough information for a single pass of the print head, processing gets interrupted and you take a big performance hit.  Remember, printers and other embedded devices don't have desktop processors in them.

I think a useful rule-of-thumb is to ensure that a client or server can supply enough data to allow useful work to be done while the next frame is coming in.  If all you have is a small fragment then that time is wasted and the user will perceive poor performance, stuttering video, etc.

---

From: Mike Belshe <[email-masked]>
Date: Tue, 8 Jul 2014 15:09:36 -0700

Thanks, Greg.

I believe your proposal is preferable to what we have today in H2.

> sent overly large ones that hurt multiplexing.

OK - thanks for clarifying.   I don't recall any such learning from SPDY.
 It certainly wasn't learning while I was at Google.

BTW, some history:
* SPDY originally used large frames too - 24bits - and let you use it as
you wished.  Some people worried that these large frame sizes would be
abused someday and felt compelled to shrink them.  But, as far as I know,
those arguments were only theoretical.  The fact is that you're really only
going to hurt yourself; and 'you' is the server.  Proxies are a different
story.

* SPDY's final field sizes were not arbitrary - they were an evolution
based on lessons of what is valuable and what works.  Initially all headers
were limited to 16k-ish.  This was because we didn't "like" large headers.
 But, when put to practice there are edge cases which needed to be dealt
with, and some apps out there, wisely or not, use ridiculously sized
headers.  It just happens.  Increasing the size was always preferable
because:
   a) we learned that the constraint itself wasn't very valuable and didn't
make anything materially better.
   b) there were edge cases where the constraint just didn't work, and the
solutions were worse than the initial problem.

> So I don't see it as a compromise - it is simply moving the limit that
> must exist from being a fixed capability of the framing layer to be an
Ok. I don't see any real need for these either.  They may introduce new
oddities if someone goes crazy with them.  I'd lean toward not adding these
if possible.

Mike

>>>    The END_STREAM flag is not present on the CONTINUATION frame, thus
>>>    it is possible for a stream to send CONTINUATION frames after a HEADERS
>>>    frame that has the END_STREAM flag set. This is confusing and increases the
>>> This proposal addresses the issue of sending/receiving large HTTP
>>> headers without giving endpoints and intermediaries unlimited resource
>>> commitments nor unknown limits
>>>
>>> The SETTINGS_FRAME_SIZE applies to all other frames including DATA
>>> frames and any other frame that may be defined by an extension. The use of
>>> this parameter is intended to tune/optimise the connection for the general
>>> case of multiple streams over the specific connection.
>>>
>>
>> Greg -
>>
>>> It is true that a 31 bit large frame length is also an arbitrary limit
>>> to the size of a frame. However, it is believed that 31 bits is
>>> sufficiently large to efficiently handle almost all conceivable present and
>>> future use cases. It would be possible to implement an unlimited size
>>> length field, but this would also need changes to the flow control
>>> mechanism, which currently also has a 31 bit maximum size.
>>>
>>> [1]
>>> http://lists.w3.org/Archives/Public/ietf-http-wg/2013AprJun/0926.html
>>> [3]
>>> http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1664.html
>>>
>>>
>>> --

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 8 Jul 2014 17:36:27 -0700

>  It certainly wasn't learning while I was at Google.

Yes, Patrick identified these issues after you stopped working on SPDY.

>    a) we learned that the constraint itself wasn't very valuable and
> didn't make anything materially better.
>> The intention of the proposal is to allow large frame sizes IF NEEDED,
>> but to use the settings to constrain implementations to the 16KB that has
>> been selected as a reasonable one-size-fits-all for todays traffic.   The
>>>> This proposal addresses the issue that a fixed frame size does not
>>>> allow tuning multiplexing performance based on current/future experience.
>>>>
>>>> There have also been issues raised that a 16KB frame size does not
>>>> allow efficient data transfer [3] even when the end points are aware that
>>>> only a single stream is likely to be required for the imminent future, or
>>>> This proposal addresses the issue of tuning the frame size for
>>>> transport efficiency for specific streams in specific situations.
>>>>
>>>> This proposal increases the length field in the frame header to 31
>>>> bits, to match the maximum flow control window size. However,
>>>> implementations will not be able to use the full frame size without
>>>> explicit consent from peers using newly defined SETTINGS or an optional
>>>> WINDOW_UPDATE field.
>>>> Frame Size Settings
>>>>
>>>> Two settings parameters are proposed: SETTINGS_HEADER_FRAME_SIZE for
>>>> the maximum header size and SETTINGS_FRAME_SIZE for all other frames.
>>>>
>>>>    - An approximate rough measure of the network latency and
>>>>    throughput - This can be derived from the timing of the receipt of the
>>>>    first few data frames and used to estimate the impact on QoS of any change
>>>>    to the maximum frame size.
>>>>
>>>> The client can use this knowledge to make an informed decision as to
>>>> the benefit of changing the allowed max frame size against any risk to
>>>>    Sufficient frame. If the remaining content is only a small
>>>>    increment over the current SETTING_FRAME_SIZE, the Max Frame Size can be
>>>>    increased to receive the remaining content in a single frame without any
>>>>    significant QoS impact.
>>>>
>>>> Minimal Compliance
>>>>
>>>> To not consider issues and proposal brought to the WG would be a fail
>>>> of the process. This proposal is based on all the hard work to date done by
>>>>    - The proposal may be able to reduce the number of frames needed
>>>>    for some content, thus saving 8 bytes. Whilst not likely to be a 25% frame

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 8 Jul 2014 17:37:56 -0700

> Yes, Patrick identified these issues after you stopped working on SPDY.

Clarification: Patrick identified actual abuses live in the wild. Not
theoretical. We indeed talked about the theoretical issue back when you
were still at Google.

>>    b) there were edge cases where the constraint just didn't work, and
>> the solutions were worse than the initial problem.
>>
>>
>>
>>>
>>>>> Whilst this educated guess may be near optimal for today's networks
>>>>> and traffic, it is entirely possible that some current and/or future
>>>>> networks may require a different value to achieve an optimal balance. There
>>>>> have already been proposals [1] put to the WG to reduce the frame size to
>>>>> This proposal is to alter the core http2 protocol to address the
>>>>> issues identified above by supporting a variable length maximum frame size
>>>>> The SETTINGS_HEADER_FRAME_SIZE parameter supports the current
>>>>> behaviour where large headers can be sent without changing the frame size
>>>>> allowed for other frame types. ie A large header size limit can be set
>>>>> without affecting the multiplexing efficiency of DATA frames.
>>>>>
>>>> Clarifying question:  are you bundling these settings parameters
>>>> because you think we need them?  Or simply as a compromise?  If you had to
>>>> choose between a larger frame size and no settings change vs the current
>>>> frame size, which would you find simpler?
>>>>
>>>> Thanks,
>>>> Mike
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>  Frame Size Updates
>>>>>
>>>>> To handle the issue of efficiently sending large data when an end
>>>>> point is prepared to risk multiplexing efficiency, this proposal allows a
>>>>> Max Frame Size to be applied to a specific stream as an optional field in a
>>>>> WINDOW_UPDATE frame.
>>>>>
>>>>>    - How many other streams are currently open and/or reserved - so
>>>>>    it knows if multiplexing is actually required.
>>>>>    No change. It can just not adjust the frame size, either because
>>>>>    it is too hard to consider or that there are too many other streams, or
>>>>>    that the content is video that needs to be received slowly. In any of these
>>>>> A minimally compliant implementation MUST handle the
>>>>> SETTING_FRAME_SIZE and SETTINGS_HEADER_SIZE and ensure that no frame sent
>>>>> exceeds the applicable limit. However no implementation is required to send
>>>>> frames at or near these limits when set above the default 16KB.
>>>>>
>>>>> *It is too late in the process to change the framing layer and to do
>>>>> so after so much discussion is an implicit fail of the WG*
>>>>>
>>>>>    - It is not yet clear if extensions will be a viable way to
>>>>>    enhance the http2 protocol. There are significant hurdle to overcome to
>>>>>    deploy extensions.
>>>>>    - Many of the issues are aimed at complexity and tuning of the
>>>>>    core protocol, and these cannot be addressed in an extension.
>>>>> Hardware acceleration is not part of the WG brief to support, nor is
>>>>> it clear that this proposal is any less suitable than others for hardware
>>>>> acceleration.
>>>>> Contributors
>>>>>

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 19:13:12 -0700

Reading through I am -1 on the frame layout changes.

I would be fine with un-reserving the first two bits and simply
requiring 64kb frames, but I feel like needing to compare allowed
sizes against SETTINGS that may change dynamically is a more
complicated change.

On Tue, Jul 8, 2014 at 5:37 PM, William Chan (陈智昌)
<[email-masked]> wrote:
> theoretical. We indeed talked about the theoretical issue back when you were
> still at Google.
>
>>
>>
>>>
>>>
>>> BTW, some history:
>>>> extensions as experience is gained, traffic changes, networks change etc.
>>>> It may well be that they are adjusted down initially as much as they are
>>>> adjusted up.
>>>>
>>>>
>>>>>> until the last frame in the sequence is processed. A receiver that wishes to
>>>>>> reject streams headers larger than a specific limit may have to process many
>>>>>> frames and hold the results in memory before it discovers the header is too
>>>>>> large.
>>>>>>
>>>>>> known in advance. The only way a sender can know if a header too large is by
>>>>>> handled in multiple frames of different types with different frame flags and
>>>>>> stream control logic. Because the vast majority of headers sent (>99.99%)
>>>>>> are below this indeterminate size, implementations will have a code path
>>>>>> that is seldom executed and probably insufficiently tested. This invites
>>>>>> poor and/or partial and/or incorrect implementation.
>>>>>>
>>>>>> arbitrary value that has been selected on the basis of experience to provide
>>>>>> a reasonable compromise between the efficiency of transmitting data vs the
>>>>>> implementations will not be able to use the full frame size without explicit
>>>>>> consent from peers using newly defined SETTINGS or an optional WINDOW_UPDATE
>>>>>> field.
>>>>>>
>>>>>> Frame Size Settings
>>>>>>
>>>>>> The content-length header - so it knows that the amount of data
>>>>>> The content-type header - so it knows if the content has high priority
>>>>>> How many other streams are currently open and/or reserved - so it
>>>>>> How many other requests are pending - so it knows if new multiplexed
>>>>>> An approximate rough measure of the network latency and throughput -
>>>>>> It is too late in the process to change the framing layer and to do so
>>>>>> after so much discussion is an implicit fail of the WG
>>>>>>
>>>>>> It is not yet clear if extensions will be a viable way to enhance the
>>>>>> Many of the issues are aimed at complexity and tuning of the core
>>>>>> It is asymmetric to support large headers with one mechanism and large
>>>>>> data with another.
>>>>>>
>>>>>>
>>>>>> The proposal may be able to reduce the number of frames needed for
>>>>>> to the size of a frame. However, it is believed that 31 bits is sufficiently
>>>>>> frame size. This proposal will not change to default behaviour of http2 with
>>>>>> regards to frame size.
>>>>>>
>>>>>>

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 19:17:02 -0700

To be clear, my issue is that this change makes determining valid
frame lengths dependent on the session state which to me seems a step
backwards both in terms of protocol layering and in terms of the
achievable decoding throughput based on the simple frame layout.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 12:33:37 +1000

Jeff,

how is a frame size limit state significantly different from the state that
must already be kept for flow control windows?   We are already stateful at
both the stream and connection level because of those and I don't think
I've seen anybody argue against flow control.

I would argue that frame size is a priori a framing layer concern and that
we already manage it dynamically for flow control, so having a dynamic
limit is not a significant nor complex change.

Note that the proposal originally had a limit for both the connection and
stream, which nicely matched flow control. Currently it is only proposed to
add a connection limit - but if I could very easily be persuaded back to
support both.

regards

On 9 July 2014 12:17, Jeff Pinner <[email-masked]> wrote:

> were
> etc.
> it
> it is
> >>>>>>> estimated duration) by increasing the frame size to something
> >16KB and <
> >>>>>>> content-length
> >>>>>>>
> increment
> >>>>>>> Optimising data transfers for large content could possibly be done
> in
> >>>>>>> an extension, however:
> >>>>>>>
> sufficiently
> >>>>>>> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client
> that
> >>>>>>> scales
> >>>>> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client
> that
> >>>>> scales

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 10:35:27 +0800

> achievable decoding throughput based on the simple frame layout.

It adds state to the framing layer, but that’s not a layering violation. Using consecutive binary-format frames to express a single semantic frame, depending on the END_HEADERS flag, is a layering violation.

Dynamic changes are unlikely to happen in practice; perhaps that should be disallowed. (If only the first SETTINGS is only allowed to increase the limit, the problem goes away.) However, safe dynamic adjustment only requires that the frame encoder applies the state change synchronously with sending the SETTINGS receipt. This is already a pattern held in common with other adjustable settings.

The frame layout is as simple as it ever was, they just increased the size from 2 to 4 bytes. Decoding is faster when frames end less frequently.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 12:56:08 +1000

On 9 July 2014 12:35, David Krauss <[email-masked]> wrote:

> Dynamic changes are unlikely to happen in practice;

If the proposal is accepted, Jetty is definitely planning on modifying our
server to adjust the frame size setting to a large value when we see a
single large upload on a request.

We will do the same for our client for downloads.

> perhaps that should be disallowed

Please no.

While I would have preferred a per stream limit for dynamic changes (to
match the per stream flow control window) as per the original proposal,
I've accepted the argument made by Martin that having that as an optional
part of the specification makes unlikely to be useful.  A dynamic setting
for the connection can be used almost the same.         But eitherway, I
think the limit should be considered as part of the flow control algorithm
and thus is naturally dynamic.

cheers

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 20:02:13 -0700

> We will do the same for our client for downloads.

Precisely why I am a strong -1 on this proposal.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 20:01:41 -0700

> It adds state to the framing layer, but that’s not a layering violation. Using consecutive binary-format frames to express a single semantic frame, depending on the END_HEADERS flag, is a layering…

I agree, which is one reason I don't like the "emit reference set at
header block fragment with END_HEADERS" rule currently in the spec.

> Dynamic changes are unlikely to happen in practice; perhaps that should be disallowed.

I fear the they will happen frequently in practice, requiring dynamic
changes to the frame encoder/decoder. In practice they will likely be
driven by what is seen at the HTTP layer (large uploads/downloads,
file-transfer, etc).

---

From: Jason Greene <[email-masked]>
Date: Tue, 8 Jul 2014 22:11:26 -0500

> driven by what is seen at the HTTP layer (large uploads/downloads,

Hey Jeff,

How is this different to you from dynamically adjusting the header table size?

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 13:13:10 +1000

> driven by what is seen at the HTTP layer (large uploads/downloads,

I'm not sure I understand the extra dynamics that you think this proposal
requires?

Currently a sender must dynamically adjust frame size by max(connection
window, stream window).  This proposal changes that to
max(connection window, stream window, settings limit).       That is hardly
a significant change.

In the receiver,  it probably does rule out the simple impl of just
allocating a 16KB and expecting a full frame to fit into it.  But such
naive impls do not actually work in practise as you invariably receive more
than one frame in your initial 16KB buffer, so subsequent frames can still
wrap over a buffer boundary.   The handling you have to do for that is no
different to the handling you have to do for large frames.  So I still
don't see any additional complexity?

So can you expand a bit more on your concern because I'm just not seeing it
(which is entirely different to not agreeing with it... which I may or may
not do once I've understood it).

cheers

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 09 Jul 2014 15:15:55 +1200

On 2014-07-09 15:02, Jeff Pinner wrote:
> On Tue, Jul 8, 2014 at 7:56 PM, Greg Wilkins wrote:
>> 
>> On 9 July 2014 12:35, David Krauss wrote:
>>> 

Keep in mind that change only means the Jetty instance emitting a large 
upload is saying it is now able to *receive* large frames.

Your implementation talking to it is under no obligations to do anything 
except take note of that, and the Jetty is still obliged to follow your 
own advertised (small?) limit when sending to you.

Amos

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 20:20:08 -0700

> How is this different to you from dynamically adjusting the header table size?

I see it differently because the compressor (at least in my
implementation and so I realize I definitely could be biased) lives
above the framing codec. The low-level framer emits "header block
fragments" to the session processing code which then emits "header
fields" to the application.

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 11:22:34 +0800

On 2014–07–09, at 10:56 AM, Greg Wilkins <[email-masked]> wrote:

> If the proposal is accepted, Jetty is definitely planning on modifying our server to adjust the frame size setting to a large value when we see a single large upload on a request.
> 

OK, I suspected I was having a failure of imagination anyway :P .

It seems pointless to only keep the frame size small only while little is being received anyway. Perhaps you intend to reduce the frame size when a request is received in the midst of a long data reception? This would trade-off HOL blocking against performance, as a single HOL block event would restore performance for subsequent requests.

I’m a bit skeptical of this scheme. The server is not responsible for (and barely capable of) fine-tuning the client’s request+upload stream. Could you confirm whether I’m thinking along the right lines?

> While I would have preferred a per stream limit for dynamic changes (to match the per stream flow control window) as per the original proposal, I've accepted the argument made by Martin that having that as an optional part of the specification makes unlikely to be useful.  A dynamic setting for the connection can be used almost the same.         But eitherway, I think the limit should be considered as part of the flow control algorithm and thus is naturally dynamic.

A per-stream setting would be nice. If I were designing from a clean slate, multiplexing would be strictly a lower layer and flow control would be a separate protocol contained within each stream. For better or worse, we still have multi-connection as a fallback.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 20:25:51 -0700

> max(connection window, stream window, settings limit).       That is hardly

The allowable data window is separate (at least in my impl) from the
frame encoder. The frame encoder has no knowledge of the window size
and it relies on the session manager to not write more data then is
allowed. The encoder simply takes a streamId and some amount of data
(and a flags on whether this data closes the stream) and writes it in
frame sized chunks.

> allocating a 16KB and expecting a full frame to fit into it.  But such naive
> impls do not actually work in practise as you invariably receive more than
> one frame in your initial 16KB buffer, so subsequent frames can still wrap
> over a buffer boundary.   The handling you have to do for that is no
> different to the handling you have to do for large frames.  So I still don't
> see any additional complexity?

The additional complexity is on checking the validity of the frame
header against current allowed size and dealing with violations.

---

From: Jason Greene <[email-masked]>
Date: Tue, 8 Jul 2014 22:29:04 -0500

> fields" to the application.

Ok so, what you don’t like then is your framing layer having to inspect a settings frame? 

If so, don’t you also have the same issue with extensions? Although I guess a key difference there is they are once per connection.

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 11:33:58 +0800

On 2014–07–09, at 11:13 AM, Greg Wilkins <[email-masked]> wrote:

> I'm not sure I understand the extra dynamics that you think this proposal requires?
> 
> Currently a sender must dynamically adjust frame size by max(connection window, stream window).  This proposal changes that to 
> max(connection window, stream window, settings limit).

Don’t forget the TCP send window, which is just as likely to be the actual limiting factor.

> In the receiver,  it probably does rule out the simple impl of just allocating a 16KB and expecting a full frame to fit into it.  But such naive impls do not actually work in practise as you invariably receive more than one frame in your initial 16KB buffer, so subsequent frames can still wrap over a buffer boundary.   The handling you have to do for that is no different to the handling you have to do for large frames.  So I still don't see any additional complexity?

If a receiver advises that the max frame size is 16KB, then it may align each frame to the front of that buffer at the time it’s processed. This is not going to be complicated by anything, or simplified except maybe by switching to UDP instead :P .

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 13:42:32 +1000

> Don’t forget the TCP send window, which is just as likely to be the actual

That is transparent to the frame size.  Implementation are not required
(and in many cases probably not able) to take tcp window size into account
when determining the size of a data frame.

> If a receiver advises that the max frame size is 16KB, then it may align
> each frame to the front of that buffer at the time it’s processed. This is
> not going to be complicated by anything, or simplified except maybe by
> switching to UDP instead :P .
>
> If a receiver wishes to implement itself in that way, then it is perfectly
able to advertise a 16KB limit in the settings and it will never see a
bigger frame.

FYI what Jetty does is to fragment data frames on arrival.  ie if we
receive only 8KB of a 16KB frame (because of TCP window or buffer wrap), we
simply pretend we've just received and 8KB data frame and forward that up
to the upper layers.   We think this is better than copying/shifting
because HTTP applications should be chunk size unaware.

cheers

---

From: Roberto Peon <[email-masked]>
Date: Tue, 8 Jul 2014 20:42:47 -0700

Allowing a large frame-size setting for any stream implies a loss of
reactivity for all other streams within that connection, so long as the
stream with the larger framesize is used.

Essentially, it adds HOL blocking in. A frame size of 2^32, would
potentially imply seconds of latency, and a failure of the protocol to
deliver adequately.

I still find it amusing (in an " I'm so sad that we keep conveniently
overlooking these things" kind of way) that we're still talking about large
frames being any kind of improvement at all in the web context when one
still needs to traverse the TLS library in order to get HTTP2 working
anyway, and when larger frame sizes cause problems with reactivity, and
thus induce latency.

Larger framesizes matter only for non-TLS connections.
That is why I'll hope that most browsers and most servers pick framesizes
that are proportional to the TLS record size, and that are relatively small
(e.g. 16k or less).

I'm happy with 16 bits of length. 24 bits of length will slow things down
marginally in the common case as we're putting 8 more '0's on the wire for
nothing almost all the time.

-=R

On Tue, Jul 8, 2014 at 8:22 PM, David Krauss <[email-masked]> wrote:

> It seems pointless to only keep the frame size small only while little is
> being received anyway. Perhaps you intend to reduce the frame size when a
> request is received in the midst of a long data reception? This would
> trade-off HOL blocking against performance, as a single HOL block event
> would restore performance for subsequent requests.
>
> I’m a bit skeptical of this scheme. The server is not responsible for (and
> barely capable of) fine-tuning the client’s request+upload stream. Could
> you confirm whether I’m thinking along the right lines?
>
> A per-stream setting would be nice. If I were designing from a clean
> slate, multiplexing would be strictly a lower layer and flow control would
> be a separate protocol contained within each stream. For better or worse,
> we still have multi-connection as a fallback.
>
>

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 13:51:28 +1000

Jeff,

> (and a flags on whether this data closes the stream) and writes it in

So it is your session manager that needs to change from
  max(connection window, stream window).
to
  max(connection window, stream window, settings limit)

you have after all described the setting as session state, so that looks
reasonable in your implementation.

> header against current allowed size and dealing with violations.

Well a naive implementation could just set a setting limit of the max frame
size it is willing to accept and then trust that it will never receive a
frame larger than that.      I know that impls should attempt to detect
such protocol violations, but I doubt few will well handle all such
violations.

I guess I accept that for a good impl, it is another parameter to check for
protocol violation and error handling is always a bit of bitch.  But I
guess I just don't see this one as prohibitive complexity given all the
other advantages.

cheers

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 09 Jul 2014 16:01:05 +1200

On 2014-07-09 15:29, Jason Greene wrote:
> On Jul 8, 2014, at 10:20 PM, Jeff Pinner wrote:
> 
>>> Hey Jeff,
>>> 
> If so, don’t you also have the same issue with extensions? Although I
> guess a key difference there is they are once per connection.

I don't see anything in the current draft section 5.5 that prevents a 
random extension being sent in a mid-connection SETTINGS frame. There is 
also no reason to doubt an extension coming along someday with an ON/OFF 
toggle meaning. Extensions are also explicitly defined as a way to 
permit future frame types. So things could get really, really messy for 
any implementation that completely separates SETTINGS extension handling 
from the framing layer.
  None of which has anything to do with the large frames proposal of 
course. Except to show the proposal is one of the simpler mechanisms in 
h2 if accepted.

Jeff, you also keep using this word "session". Do you mean stream? or 
connection? or are you conflating HTTP/2 with classical UA sessions 
(sets of streams)?

Amos

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 12:03:41 +0800

Simultaneous reply to these two messages:

On 2014–07–09, at 11:42 AM, Greg Wilkins <[email-masked]> wrote:

> That is transparent to the frame size.  Implementation are not required (and in many cases probably not able) to take tcp window size into account when determining the size of a data frame.

On 2014–07–09, at 11:42 AM, Roberto Peon <[email-masked]> wrote:

> Allowing a large frame-size setting for any stream implies a loss of reactivity for all other streams within that connection, so long as the stream with the larger framesize is used.
> 
> Essentially, it adds HOL blocking in. A frame size of 2^32, would potentially imply seconds of latency, and a failure of the protocol to deliver adequately.

HOL blocking is the sender’s own fault for getting ahead of itself. If the sender doesn’t know how much it can instantaneously send due to a failure of the underlying TCP implementation, it should set a reasonable limit.

If attempting to fill a ridiculous frame limit isn’t obviously a bad idea, then the spec might recommend not sending more than 16 or 32K unless the sender knows what they’re doing. (Many folks still deal with 16-32K/sec on a regular basis!)

The possibility of misbehavior shouldn’t constrain us to use the graceful-degradation case *all the time*. Yet, every implementer should respect robustness over performance.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 14:05:15 +1000

> potentially imply seconds of latency, and a failure of the protocol to

Indeed, that is why the proposal includes setting limits that should only
be increased when it is known that QoS will not be affected or is not
important.   Nobody is saying that sending 2^31-1 frames will not hurt QoS.

> Larger framesizes matter only for non-TLS connections.

And TLS is very frequently offloaded and will be more often if the browsers
insist on making it mandatory.

So large framesizes may only be applicable for traffic within the data
centre, but any such workload I can get off my server the better!

Even for dynamic generated content from JSPs, Servlets etc. we have found
that it is far more scalable to give them a nice big 64KB or 128KB buffer
so they can run to completion without blocking and we can then flush that
buffer asynchronously.

It is also not yet given if TLS only deployment of h2 will be widely
accepted.

regards

---

From: Roberto Peon <[email-masked]>
Date: Tue, 8 Jul 2014 21:07:39 -0700

It wasn't a matter of acceptance-- it was a matter of reliability.

-=R

On Tue, Jul 8, 2014 at 9:05 PM, Greg Wilkins <[email-masked]> wrote:

> And TLS is very frequently offloaded and will be more often if the
> browsers insist on making it mandatory.
>

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 8 Jul 2014 21:21:33 -0700

> connection? or are you conflating HTTP/2 with classical UA sessions (sets of
> streams)?

The layering as I implement it:

Framing (encoding/decoding)
Session (stream state, shared connection state))
HTTP (frame mapping, semantics)

So by session here I mean the set of stream state (priority, flow
control, etc.) and connection state shared between streams (session
flow control, header compression).

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 12:25:45 +0800

On 2014–07–09, at 11:42 AM, Roberto Peon <[email-masked]> wrote:

> That is why I'll hope that most browsers and most servers pick framesizes that are proportional to the TLS record size, and that are relatively small (e.g. 16k or less).

Initial implementations of multiplexers have ample opportunity to shoot themselves in the foot.

The naive way to write a mux is to encode data into the output stream as it arrives from the application threads. Queueing a bunch of short frames adds the same HOL as one big frame. Maybe my intuition about naiveté is wrong, but:

1. An implementation that makes any decision about how much should be enqueued will apply that to the frame size,
2. An implementation that always tries to send as much as possible will incur HOL regardless of the binary frame format,
3. Any workable mux that encounters HOL in the field easily can be fixed by setting an internal limit, likely a hard limit, unless conditions indicate that large send is safe.

We shouldn’t get hung up on what the binary format seems to suggest to programmers. It’s a grammar, and its function is to provide expressive power. Even the simplest languages may express obscenity; censorship is futile. IMHO, frame size limits only feel important now because we’re debating it.

> I'm happy with 16 bits of length. 24 bits of length will slow things down marginally in the common case as we're putting 8 more '0's on the wire for nothing almost all the time.

If we were this nitpicky, the WINDOW_UPDATE frame headers could be halved.

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 9 Jul 2014 15:11:30 +1000

> that are proportional to the TLS record size, and that are relatively small
Don't forget that some of us are going to be using IE a lot more in future,
if that lets us use HTTP/2 without TLS.

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 09 Jul 2014 17:10:43 +1200

> potentially imply seconds of latency, and a failure of the protocol to

On a 100Gbps network link the transfer time of 4GB (2^32) is approx. 
0.32 seconds.

The same large frame sent as HEADERS+CONTINUATION causes identical 
problems while also expanding the total transfer size by +2MB of 
CONTINUATION frame headers. That is extra 1.5ms latency at 10Gbps and 
much more at any lower speeds.

The same large frame sent as DATA is:
* uni-directional and negotiated. In order to deliver the recipient has 
already advertised willingness to be blocked by it.
* 2MB smaller than the equivalent set of 16KB DATA frames. (shaves off
* the receiver implementation is free to send back smaller frames and 
control frames
* the receiver is free to re-package as smaller DATA frames for 
tighter/slower next-hop connections.

Also, HTTP traffic wanting such huge frames is in the minority of 
use-cases. Intra/inter -datacenter, "big data", and science networks 
primarily. The take away point being that they do exist already.

> anyway, and when larger frame sizes cause problems with reactivity, and

Correction: we are talking about larger than *16KB* as being useful for 
some of todays network needs and very useful for some near-future 
projected needs.

You are also omitting the 91% of HTTP traffic which is non-TLS. Some 
portion of which will never become TLS encrypted.

> that are proportional to the TLS record size, and that are relatively 

There is effort ongoing to increase the throughput and hardware 
acceleration in the encryption area. We may find that in only a few 
years it is possible to send Gbps out through a hardware encrypting NIC. 
There is also significant ongoing efforts to improve the library basic 
throughputs.

It all comes back to implementations needing to be free to pick the 
right frame sizes for their use-cases. Large or small.

Amos

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Jul 2014 14:50:00 +0800

On 2014–07–09, at 1:10 PM, Amos Jeffries <[email-masked]> wrote:

> There is effort ongoing to increase the throughput and hardware acceleration in the encryption area. We may find that in only a few years it is possible to send Gbps out through a hardware encrypting NIC.

Is this not already commonplace?

http://www.cavium.com/solution_ssl_switch.html
http://www.cavium.com/Adapters_Crypto_Offload.html

A high-end model can decrypt (and decompress, and do the reverse) 60 Gbps. If that sounds like a lot, it’s because it’s a lot of dedicated silicon.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 8 Jul 2014 23:52:33 -0700

Sure, good luck with that 85% success rate :)
Makes sense on an intranet. Not so much on the wild, wild internet, unless
things have substantially changed.
-=R

On Tue, Jul 8, 2014 at 10:11 PM, Matthew Kerwin <[email-masked]>
wrote:

> Don't forget that some of us are going to be using IE a lot more in
> future, if that lets us use HTTP/2 without TLS.
>
> --
>   Matthew Kerwin

---

From: <[email-masked]>
Date: Wed, 9 Jul 2014 10:30:16 +0000

Hi Roberto-

On Wednesday,09 July 2014 08:53, [email-masked] wrote:
> On Tue, Jul 8, 2014 at 10:11 PM, Matthew Kerwin <[email-masked]> wrote:
>> Don't forget that some of us are going to be using IE a
>> lot more in future, if that lets us use HTTP/2 without TLS.

We likely fall into that category as well.

> Makes sense on an intranet. Not so much on the wild,
> wild internet, unless things have substantially changed.
> -=R

Success rate of what?  Are you referring to IE?  Does that browser have a particular success rate issue?  Or are you referring to an issue with clear-text HTTP?  Clearly I am missing some context.  If this was already discussed on-list and you can just point me to the discussion I'll gladly go read it.

Thanks.

-keith

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Jul 2014 08:10:36 -0400

> stream with the larger framesize is used.

yes. "all other" plus "all future" streams.. cancel is a big deal here.

> potentially imply seconds of latency, and a failure of the protocol to
yes. Most of the value of h2 is in prioritized mux - it makes sense that
the protocol operates in a way that makes that work. If we could figure out
how to disable buffering I would support that too :)

greg's proposal is a worthwhile extension for those that are interested.
We made the same extension decision for alt-svc even though that also deals
with a core property of the protocol.

> anyway, and when larger frame sizes cause problems with reactivity, and
yes.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 09 Jul 2014 12:16:33 +0000

In message <[email-masked]>, Patrick McManus 

>> stream with the larger framesize is used.

Only if the allowance for larger frames is being exploited by the sender.

Nobody forces the sender to send packets any larger than it wants to.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Jul 2014 08:26:22 -0400

> Nobody forces the sender to send packets any larger than it wants to.

obviously if jumbo frames are not used jumbo frames dont matter

but when they are used, all of the receivers streams are put at risk. So
the client is incented to use multiple connections so the risk is not
linked. And that's a fail for h2.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 09 Jul 2014 12:32:17 +0000

In message <CAOdDvNptJiAUcr1sShO6dbfKNya9xm8TTkqc7fk7-cYy=[email-masked]>, Patrick McManus 

>linked. And that's a fail for h2.

First, this is rather trivially detectable by any moderately compehensive DoS
detection scheme.

Second, what you describe amounts to a client using HTTP/2 in HTTP/1 fashion.

That would be the almost trivial way to add HTTP/2 support to many
HTTP/1 implementation, without having to fundamentally change the
applications internal scheduling.

It may aid HTTP/2 adoption if we are not too harsh too soon on this "hack".

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Jul 2014 22:41:35 +1000

> linked. And that's a fail for h2.

Patrick,

I don't see the incentive to use multiple connections.   If an endpoint
wants streams to proceed in parallel, then it should not send nor accept
large frames.    We have already shown that going multiplexed over a single
connection has advantages over multiple connections, but if an endpoint
really wanted multiple connections with large frames, then it would use h1
rather than pervert h2.

Besides, I don't think this is a black and white issue here.   Firstly, it
is not given that increases in frame sizes will be to huge frames, as it
maybe that for a given network/situation 32KB frames give good throughput
and acceptable QoS.    More over, it is simply not the case that 16KB
frames give good QoS and any thing greater does not.     The proposal can
also be used to decrease frame sizes if that is optimal for some situations.

We have already accepted that endpoints have to be restrained with large
headers if they want QoS, so we are already trusting them with that
decision.  How is this any different for large data?

regards

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Jul 2014 11:18:10 -0400

On Wed, Jul 9, 2014 at 8:41 AM, Greg Wilkins <[email-masked]> wrote:

> I don't see the incentive to use multiple connections.
>

If I have N transactions on a connection that uses jumbo frames then all N
transaction have a responsiveness risk anytime a jumbo frame is used
because the server cannot react to a CANCEL (or other high priority event)
without serializing the entire jumbo frame.  If I start a 500MB download
and then cancel it and start a different 500MB download the subsequent
transfer cannot happen in a timely fashion if the first one used a 500MB
frame and they all share the same connection.

There are two fixes to that problem:

1] don't use jumbo frames. that's the status quo. Its a good thing - but
the preface above was "when they are used"

2] place N transactions on N connections the way h1 does. The stream cancel
turns into a tcp rst which gives the appropriate level of reactivity.
Unfortunately now I'm not running a prioritized muxed protocol with good
connection handling - that is a bad thing :)

I appreciate that the proposal makes frames larger than 16KB optional for
the receiver and I appreciate it. My objection is that you can't create
large frames at all and have it be compatible with a prioritized mux'd
protocol - so it shouldn't be there because there really is no other reason
to want h2 other than priority mux and large frames are inherently not
priority mux friendly. I also appreciate that it allows mini frames (is
that the opposite of a jumbo frame?) but at this point in the schedule it
doesn't seem to be addressing a demonstrated problem, so I'd rather proceed
on the LC call track with this as an extension.

-P

> If an endpoint wants streams to proceed in parallel, then it should not
> send nor accept large frames.    We have already shown that going
> multiplexed over a single connection has advantages over multiple
> connections, but if an endpoint really wanted multiple connections with
> large frames, then it would use h1 rather than pervert h2.
>

---

From: James M Snell <[email-masked]>
Date: Wed, 9 Jul 2014 08:37:36 -0700

Ok, so we do both. Set the default operation of the protocol to
size-16k frames. If an endpoint wishes to allow jumbo frames, it
enables an extension that says "enable jumbo frames. yes, yes, I know
that means the protocol will be less responsive, and I get that, but I
have good reason to allow for it so, please, go ahead and do it". Once
that setting is set, there's no going back on that connection and the
willful participants go into it knowing full well that the protocol
will be less responsive for multiplexed streams. The folks who are
willfully enabling the switch will be well aware of those "pitfalls"
and will understand what they are doing and why.

On Wed, Jul 9, 2014 at 8:18 AM, Patrick McManus <[email-masked]> wrote:
> transaction have a responsiveness risk anytime a jumbo frame is used because
> the server cannot react to a CANCEL (or other high priority event) without
> serializing the entire jumbo frame.  If I start a 500MB download and then
> cancel it and start a different 500MB download the subsequent transfer
> cannot happen in a timely fashion if the first one used a 500MB frame and
> they all share the same connection.
>
> 1] don't use jumbo frames. that's the status quo. Its a good thing - but the
> preface above was "when they are used"
>
> priority mux friendly. I also appreciate that it allows mini frames (is that
> the opposite of a jumbo frame?) but at this point in the schedule it doesn't
> seem to be addressing a demonstrated problem, so I'd rather proceed on the
> LC call track with this as an extension.
>
> -P
>
>
>
>
>>
>> and acceptable QoS.    More over, it is simply not the case that 16KB frames
>> give good QoS and any thing greater does not.     The proposal can also be
>> used to decrease frame sizes if that is optimal for some situations.
>>

---

From: Jeff Pinner <[email-masked]>
Date: Wed, 9 Jul 2014 09:16:34 -0700

> On a 100Gbps network link the transfer time of 4GB (2^32) is approx. 0.32
> seconds.
>
> The same large frame sent as HEADERS+CONTINUATION causes identical problems
> while also expanding the total transfer size by +2MB of CONTINUATION frame
> headers. That is extra 1.5ms latency at 10Gbps and much more at any lower
> speeds.

The transfer may take 0.32 seconds on a 100Gbps network, but on that
same link, the overhead from CONTINUATION frame headers is only .00015
seconds (.04%)

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 9 Jul 2014 10:15:23 -0700

On Wed, Jul 9, 2014 at 3:30 AM, <[email-masked]> wrote:

> Success rate of what?  Are you referring to IE?  Does that browser have a
> particular success rate issue?  Or are you referring to an issue with
> clear-text HTTP?  Clearly I am missing some context.  If this was already
> discussed on-list and you can just point me to the discussion I'll gladly
> go read it.
>

The success rate is HTTP Upgrade in cleartext over the web as tested with a
single Google server and Google Chrome clients in an experiment. And 85%
was for a separate port. For port 80, it was 63%. Details here:
http://www.ietf.org/mail-archive/web/tls/current/msg05593.html. More
general analysis at my blog:
https://insouciant.org/tech/http-slash-2-considerations-and-tradeoffs/#Upgrade,
including discussions of other deployment options and their success rates.

>
> Thanks.
>
> -keith
>
> This email message is intended only for the use of the named recipient.
> Information contained in this email message and its attachments may be
> privileged, confidential and protected from disclosure. If you are not the
> intended recipient, please do not read, copy, use or disclose this
> communication to others. Also please notify the sender by replying to this
> message and then delete it from your system.
>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 09 Jul 2014 20:44:50 +0200

On 2014-07-09 19:15, William Chan (陈智昌) wrote:
> On Wed, Jul 9, 2014 at 3:30 AM, <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi Roberto-
>
>     On Wednesday,09 July 2014 08:53, [email-masked]
>     <mailto:[email-masked]> wrote:
>      > On Tue, Jul 8, 2014 at 10:11 PM, Matthew Kerwin
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     Success rate of what?  Are you referring to IE?  Does that browser
>     have a particular success rate issue?  Or are you referring to an
>     issue with clear-text HTTP?  Clearly I am missing some context.  If
>     this was already discussed on-list and you can just point me to the
>     discussion I'll gladly go read it.
>
>
> The success rate is HTTP Upgrade in cleartext over the web as tested
> with a single Google server and Google Chrome clients in an experiment.
> And 85% was for a separate port. For port 80, it was 63%. Details here:

It would be interesting to repeat that experiment. It's now 4.5 years 
later, and deploying Websockets may have caused broken code to be fixed.

Best regards, Julian

---

From: David Krauss <[email-masked]>
Date: Thu, 10 Jul 2014 07:01:59 +0800

On 2014–07–09, at 11:18 PM, Patrick McManus <[email-masked]> wrote:

> If I have N transactions on a connection that uses jumbo frames then all N transaction have a responsiveness risk anytime a jumbo frame is used because the server cannot react to a CANCEL (or other high priority event) without serializing the entire jumbo frame.  If I start a 500MB download and then cancel it and start a different 500MB download the subsequent transfer cannot happen in a timely fashion if the first one used a 500MB frame and they all share the same connection.
> 
> 1] don't use jumbo frames. that's the status quo. Its a good thing - but the preface above was "when they are used"
> 
> 2] place N transactions on N connections the way h1 does. The stream cancel turns into a tcp rst which gives the appropriate level of reactivity. Unfortunately now I'm not running a prioritized muxed protocol with good connection handling - that is a bad thing :)
> 
> I appreciate that the proposal makes frames larger than 16KB optional for the receiver and I appreciate it. My objection is that you can't create large frames at all and have it be compatible with a prioritized mux'd protocol

Please see my recent messages to the list. There is a fundamental misunderstanding about what it means to be able to generate large frames. Just because someone can, doesn’t mean they should. The standard should probably make this clear, since it seems to be such a point of misunderstanding.

The problem of writing a working mux is exactly the same with and without jumbo frames. They simply allow removal the special case handling of multi-frame header blocks, and reduce the inherent per-frame CPU overhead when correctly availed of. A mux should make some decision about how much to write to its output stream before it calls write(). Blocking on write() is a Bad Thing and will likely result in mismatched HTTP and IP frames. Don’t do it.

Given that that is being avoided, an frame size limit simply is not needed to dictate mux behavior.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 9 Jul 2014 16:51:59 -0700

On Wed, Jul 9, 2014 at 4:01 PM, David Krauss <[email-masked]> wrote:

> > If I have N transactions on a connection that uses jumbo frames then all
> N transaction have a responsiveness risk anytime a jumbo frame is used
> > 2] place N transactions on N connections the way h1 does. The stream
> cancel turns into a tcp rst which gives the appropriate level of
> reactivity. Unfortunately now I'm not running a prioritized muxed protocol
> with good connection handling - that is a bad thing :)
> >
> > I appreciate that the proposal makes frames larger than 16KB optional
> for the receiver and I appreciate it. My objection is that you can't create
> protocol
> Please see my recent messages to the list. There is a fundamental
> misunderstanding about what it means to be able to generate large frames.
> Just because someone can, doesn’t mean they should. The standard should
> probably make this clear, since it seems to be such a point of
> misunderstanding.
>

I suspect we're agreed on that :)

>
> The problem of writing a working mux is exactly the same with and without
> jumbo frames. They simply allow removal the special case handling of
> multi-frame header blocks, and reduce the inherent per-frame CPU overhead
> when correctly availed of. A mux should make some decision about how much
> to write to its output stream before it calls write(). Blocking on write()
> is a Bad Thing and will likely result in mismatched HTTP and IP frames.
> Don’t do it.
>
>
Not really true, though.
Lets say I write a muxing sender which looks at the available streams,
selects the highest priority amongst them, then generates a frame for it
and writes it to the socket.

With a small framesize, that'll work just fine.
With a large framesize, that won't work. You may end up with single frames
which take seconds or even hours (4GB of data on a low bandwidth link, for
instance) to drain.

This is orthogonal to the quality of the rest of the implementation.

> Given that that is being avoided, an frame size limit simply is not needed
True because a well implemented mux would select a small frame size.
In other words a (small) frame size is required to achieve good mux
behavior. A limit on the max framesize (dynamic or not) is one way to
accomplish that.

We've seen server implementations of SPDY get this wrong (when all that was
necessary was to fragment responses into smaller frames).
We have at least one declaration that folks would use larger frame sizes
for uploads or downloads, either of which will reduce the reactivity of the
session and cause a failure of the protocol to achieve its goals for that
connection if anything else happens during that upload/download.

-=R

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 10 Jul 2014 10:07:06 +1000

Patrick,

that is not in dispute.   But there are two mitigations:

   - You are suggesting that the only way this proposal can be used in this
   example is to send a 500MB frame.  That is not the case, as the flow
   control window at least will limit frames to 64KB initially.   Even once
   the flow control window is enlarge, it is not certain that a 500MB frame
   will be used.  The printer use case has indicated that simply doubling the
   frame size to 64KB is sufficient for a significant increase in
   throughput.       We get that HUGE frames are bad, but that should not mean
   that we prohibit larger frames.

   - if somebody starts sending a 500MB header (in either 1 or more frames)
   then we have the same issue. So our QoS is already entirely dependent on
   the good sense of a single endpoint.   Large or larger frames relies on the
   good sense of both end points.

At one stage I was not too bothered with LC on h2-13, but now that not
implementing continuations is seen as being a bad actor, I can't live with
a LC on h2-13, regardless of timetables.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 10 Jul 2014 10:21:29 +1000

> which take seconds or even hours (4GB of data on a low bandwidth link, for

Again with this strawman analysis!

It is not a choice between small==good and large==bad.    There is a
spectrum of possible frame sizes and picking a single arbitrary 16KB frame
size is unlikely to be optimal in even a majority of today's networks.
This proposal is to allow 32KB, 64KB 128KB frames much more than it is to
occasionally allow a much larger frame.  In fact we have removed the
WINDOW_UPDATE part of the proposal that would facilitate easy sending of
very large frames for large content.

it would be interesting to perform an experiment on a fast network and
compare user experience between multiplexed  traffic over 16KB frames vs
64KB frames.   I'm betting that most users will not notice the difference
in terms in QoS/interruptability etc. but they will notice the faster down
load times of images etc. with the large frame size.

Regardless, the argument against this proposal based of the fact that
sending a 2GB frame will break multiplexing is a straw man.  Just because a
mechanism can be misused is not a sufficient argument against it.   In this
case, large frames requires the collaboration of both peers, while large
headers can currently be sent by a single peer (Even if it will get 431'd)

regards

---

From: Roberto Peon <[email-masked]>
Date: Wed, 9 Jul 2014 17:40:46 -0700

On Wed, Jul 9, 2014 at 5:21 PM, Greg Wilkins <[email-masked]> wrote:

>> With a large framesize, that won't work. You may end up with single
>> frames which take seconds or even hours (4GB of data on a low bandwidth
>> link, for instance) to drain.
>>
>

I'm happy with 64k as a max-representable framesize :)
128k is almost OK-- I'd just steal a bit from the type field.

I didn't personally mind the window-update part of the proposal, btw. I
thought it was reasonable given that negative windows already need
handling. *shrug*

> headers can currently be sent by a single peer (Even if it will get 431'd)

I'm fairly sure it isn't a strawman since I'm talking about evidence of
misuse of the capability of sending large frames as evidenced in SPDY :)
Regardless, I'm saying that the current framesize was a direct reaction to
that experience and evidence.

Lets assume that we're OK with larger frame sizes.
At that point the question becomes: How large should the length field be?
Imho, 64k for a frame is plenty large: 2048 frames/gig seems cheaply doable.

I've said it elsewhere, in other ways but:
I'm down with a larger max framesize (within limits: I feel that even 2^24
is pushing it).
I'm supportive of a setting for max framesize.
-=R

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 10 Jul 2014 10:47:07 +1000

On 10 Jul 2014, at 10:40 am, Roberto Peon <[email-masked]> wrote:

> I'm down with a larger max framesize (within limits: I feel that even 2^24 is pushing it).

There's another aspect here -- if we have larger frames, are people OK with the max frame size becoming the effective ceiling on compressed header block size, because CONTINUATION is ditched in the process?

Thanks,

---

From: Roberto Peon <[email-masked]>
Date: Wed, 9 Jul 2014 17:54:12 -0700

On Wed, Jul 9, 2014 at 5:47 PM, Mark Nottingham <[email-masked]> wrote:

> There's another aspect here -- if we have larger frames, are people OK
> with the max frame size becoming the effective ceiling on compressed header
> block size, because CONTINUATION is ditched in the process?
>
> Thanks,
>
>

That can become very dangerous.
If there is a single limit on framesize, then it would end up being large
because the receiving party is unlikely to know how large the headers
should be, and must set it to as large as they ever believe headers might
be.
That negatively impacts reactivity.

One could fix this by have separate headers vs data max frame size.
Then we'd need to figure out which one mapped to SETTINGS and other control
frames.
Would it be a control-frame vs data-frame setting?
Bleh.

-=R

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 10 Jul 2014 10:57:16 +1000

On 10 Jul 2014, at 10:54 am, Roberto Peon <[email-masked]> wrote:

> If there is a single limit on framesize, then it would end up being large because the receiving party is unlikely to know how large the headers should be, and must set it to as large as they ever believe headers might be.
> Then we'd need to figure out which one mapped to SETTINGS and other control frames.

That's the direction that greg et al have proposed, and it seems to have good support.

> Bleh.

I need a better reason than "bleh" to justify the decision to the IESG...

Cheers,

---

From: Roberto Peon <[email-masked]>
Date: Wed, 9 Jul 2014 18:03:10 -0700

Will double-bleh do?

Nope? I thought not.

The proposal will work just fine so long as you don't get a pair of entites
on the path that do the stupid together.
Hopefully this isn't something that would happen today, but who knows.
With a smaller max frame size and continuations, you need implementations
that screwed up by more than just setting a single variable too big in
order to screw with latency.

So, how much do we want to empower folks to shoot themselves in the head in
order to deal with the 0.02% usecase?

-=R

On Wed, Jul 9, 2014 at 5:57 PM, Mark Nottingham <[email-masked]> wrote:

> > If there is a single limit on framesize, then it would end up being
> large because the receiving party is unlikely to know how large the headers

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 10 Jul 2014 11:48:09 +1000

On 10 July 2014 10:40, Roberto Peon <[email-masked]> wrote:

>> It is not a choice between small==good and large==bad.   ...
>>
>

So max frame size is a little bit of personal preference mixed in with some
experience gained from SPDY (on todays networks with todays traffic).

I expect opinions and experience are going to change over the next 10 to 15
years.

So the point of the proposal is to move the limit from being a fixed
capacity of the frame format to a settings that can adapt to preferences,
fashion, direction of the wind, whatever.      While 64KB may well be
better than 16KB today, I don't think we can say it will always be so in
all situations.    Having the limit as a setting is just more flexible.

> thought it was reasonable given that negative windows already need
I still think it is a good idea.... but discussion about it was a
distraction for the more important changes in the proposal.  Martins point
about it not being that worthwhile if optional is also a good one.
I'm happy to re-propose it as a separate proposal once larger frames are
accepted.

> misuse of the capability of sending large frames as evidenced in SPDY :)

The strawman is to say that because you can send huge frames that they will
be sent in situations that will break multiplexing therefore this proposal
breaks multiplexing.   I see it as a strawman because a bad actor can
already send HEADERS+CONTINUATION* of arbitrary size an a good actor has an
incentive to not break multiplexing.

> that experience and evidence.

which is really great, and thus 16KB is the default value suggested for the
limit.   But that direct experience and evidence will not be correct for
all situations nor for all time.     Let's not lock in one snapshot of
traffic into the base capabilities of the framing layer.

> I'm supportive of a setting for max framesize.

I think we are essentially in agreement.   I see 2^31 frame sizes as
essentially infinite and yes 64KB or 128KB max frame sizes will almost
certainly give lots of wiggle room.   But setting a hard limit at just
above todays optimised max frame size is asking for trouble in the future.

I think of 2^31 as infinite size and I don't expect anybody to ever send
infinite frames.   The limit is in the settings not in the frame format.

cheers

---

From: Jason Greene <[email-masked]>
Date: Wed, 9 Jul 2014 21:36:14 -0500

On Jul 9, 2014, at 8:03 PM, Roberto Peon <[email-masked]> wrote:

> Will double-bleh do?
> 
> Nope? I thought not.
> 
> The proposal will work just fine so long as you don't get a pair of entites on the path that do the stupid together.
> With a smaller max frame size and continuations, you need implementations that screwed up by more than just setting a single variable too big in order to screw with latency.

For this to be accurate continuations would have to be improved to allow multiplexing. If they don’t they are equivalent to a large jumbo frame, just written in chunks.

> 
> So, how much do we want to empower folks to shoot themselves in the head in order to deal with the 0.02% usecase?
> 
> -=R

---

From: Jeff Pinner <[email-masked]>
Date: Wed, 9 Jul 2014 21:13:24 -0700

> experience gained from SPDY (on todays networks with todays traffic).

experience from SPDY that 24-bits is too large plus a preference for
byte boundaries = 64k

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 10 Jul 2014 06:22:04 +0000

In message <[email-masked]>, Mark Nottingham writes:

>with the max frame size becoming the effective ceiling on compressed 
>header block size, because CONTINUATION is ditched in the process?

I'm fine with a 64KB max possible (but subject to SETTINGS) size of
compressed HEADERS.

In fact I'd really hate it if we opened for megabyte HEADERS and had to
cope with them in the future.

This also slightly mitigaes the "HEADERS not subject to window" issue.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 10 Jul 2014 06:25:43 +0000

In message <CA+pLO_hKYi=[email-masked]>, Jeff Pinner writ
es:
>experience from SPDY that 24-bits is too large  [...]

too large for what and when ?

For max frame size, I don't think 16 or 17 bits is enough.

For max header size I think 64 KB is more than plenty.

---

From: <[email-masked]>
Date: Thu, 10 Jul 2014 09:49:27 +0000

To quote Mark from an earlier thread...

On Wednesday,25 June 2014 06:11, [email-masked] wrote:

> WRT the "jumbo" frame (i.e., flagging that some prefix of the

> payload is actually an extension length) -- this sort of hack is

> necessary to back-port bigger frames onto an existing protocol

Amen. A 31-bit frame-length field may be too many bits today, but IMO we shouldn't set ourselves up for an ugly hack later.

So I propose some variant of the following...

  0                   1                   2                   3

  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|   Reserved    |                  Length (24)                  |

+-+-------------+-----------------------------------------------+

|R|                 Stream Identifier (31)                      |

+-+-------------+---------------+-------------------------------+

|   Type (8)    |   Flags (8)   |

+===============+===============+===============================+

|                   Frame Payload (0...)                      ...

+---------------------------------------------------------------+

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 10 Jul 2014 10:18:07 +0000

In message <[email-masked]>, [email-masked] w
rites:

>Amen. A 31-bit frame-length field may be too many bits today, but IMO we 
>shouldn't set ourselves up for an ugly hack later.
>

Works for me.

---

From: <[email-masked]>
Date: Thu, 10 Jul 2014 10:33:49 +0000

Jeff said:

> To be clear, my issue is that this change makes determining valid frame

> lengths dependent on the session state which to me seems a step

Jeff also said:

> I fear [changes to SETTINGS_FRAME_SIZE] will happen frequently in

> practice, requiring dynamic changes to the frame encoder/decoder.

We keep repeating that SETTINGS_FRAME_SIZE is a *non-obligatory* message from an endpoint to tell its peer it is *willing* to accept *larger* frames - which is a true statement, an endpoint is never required to send frames larger than 16K.

However, with respect to Jeff's thoughts above, I can see how he might think that dynamic changes to SETTINGS_FRAME_SIZE would require dynamic changes to the frame encoder/decoder.  What happens if the peer reduces the SETTINGS_FRAME_SIZE below 16K, is the endpoint *required* to send *smaller* frames?

The *intent* was that SETTINGS_FRAME_SIZE is *non-obligatory* whether greater or smaller than 16K; a suggestion if you will.  Hence the following requirement in the 'Frame Size' section:

By default, the maximum frame payload size is 2^14 (16,384) octets.

All implementations MUST be configurable to handle frames up to this size correctly.

The default frame payload size is optimized for stream multiplexing and ought to be

used for most cases.

We should probably make this more clear though, particularly in the description of SETTINGS_FRAME_SIZE (&SETTINGS_HEADER_FRAME_SIZE).

This is important because it allows an implementation, like Jeff's, to use a fixed 16K frame size and not worry about dynamic changes to min/max allowed frame lengths.

I suspect many limited-resource embedded implementations would also benefit from a fixed frame size.

(Although hardware acceleration is not part of the charter, I think this property may also be important for HW acceleration as it may be more efficient for HW to deal with a fixed frame size.)

-keith

---

From: Yoav Nir <[email-masked]>
Date: Thu, 10 Jul 2014 13:47:17 +0300

On Jul 10, 2014, at 12:49 PM, [email-masked] wrote:

> +———————————————————————————————+

Is that for every frame, or just when a large frame size has been advertised?  I guess the former, but then we’re increasing every frame by 2 bytes for the same of those 0.02% No?

Yoav

---

From: <[email-masked]>
Date: Thu, 10 Jul 2014 11:00:01 +0000

Hi Yoav-

On Thursday,10 July 2014 12:47, [email-masked] wrote:
> Is that for every frame, or just when a large frame size has been advertised?
> I guess the former, but then we're increasing every frame by 2 bytes for the
> same of those 0.02% No?

Our proposal (Greg et al) already had the extra 2 bytes from the beginning, see the first e-mail (from Greg) in this thread [1].  I simply extended the reserved bits from 1 to 8 to appease concerns by Roberto and others that too many bits is too big of a foot gun, but simultaneously allows future revs of the protocol to *easily* extend the frame length field if necessary.

Our proposal (Greg et al) is *not* just about getting rid of CONTINUATION frames for the 0.02%.  Again, see the first e-mail (from Greg) in this thread [1] for a _detailed_ analysis of the benefits of this proposal.  It's probably better to comment directly on sections of that e-mail if you have concerns.

Thanks.

-keith

[1] http://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/0427.html

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Thu, 10 Jul 2014 21:32:32 +0900

On Thu, Jul 10, 2014 at 8:00 PM, <[email-masked]> wrote:

> Our proposal (Greg et al) already had the extra 2 bytes from the
> beginning, see the first e-mail (from Greg) in this thread [1].  I simply
> extended the reserved bits from 1 to 8 to appease concerns by Roberto and
> others that too many bits is too big of a foot gun, but simultaneously
> allows future revs of the protocol to *easily* extend the frame length
> field if necessary.
>
>
If reserved bits are for future use for larger frame size, why not reserve
16 bits for now?
It makes a foot gun even less likely​.  I heard that many said that 16 bits
length is enough for today.

With 16 bits available (2 bits extra from h2-13's 14 bits), we can say good
bye to CONTINUATION.  This solves #549 and #550 (and possibly #551 for <
64KB, but I think it is enough).  Extra 16 reserved bits is a possible
solution of #553.

Best regards,
Tatsuhiro Tsujikawa

> Our proposal (Greg et al) is *not* just about getting rid of CONTINUATION
> frames for the 0.02%.  Again, see the first e-mail (from Greg) in this
> thread [1] for a _detailed_ analysis of the benefits of this proposal.
>  It's probably better to comment directly on sections of that e-mail if you
> have concerns.
>
> Thanks.
>
> -keith
>
>

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 11 Jul 2014 00:37:37 +1200

> experience from SPDY that 24-bits is too large plus a preference for

SPDY had end-to-end 24-bits by default. As I understand it some bad
actors could and did force their large frames down unwilling
participants connections and got away with it.

The proposal is to have hop-by-hop agreement for anything other than
16KB. With bad actors receiving 431 responses, whether they be
application layer actors or h2 implementations.

Totaly different behaviour incentives, designed explicitly to encourage
a healthy HTTP/2 environment.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 11 Jul 2014 00:52:37 +1200

On 10/07/2014 10:18 p.m., Poul-Henning Kamp wrote:
> [email-masked] writes:
> 

WFM.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 11 Jul 2014 00:52:01 +1200

> 64KB, but I think it is enough).  Extra 16 reserved bits is a possible

So in effect, adding 16 bits of reserved space before the length. THen
defining that when the SETTINGS_HEADER_FRAME_SIZE extension setting is
received is un-reserves 8 or 16 of those bits and the become a longer
length value.

Lets call a spade a spade:
  24/31-bit length field with a default maximum size of 16KB.

Amos

---

From: <[email-masked]>
Date: Thu, 10 Jul 2014 13:23:44 +0000

On Thursday,10 July 2014 14:52, [email-masked] wrote:
> So in effect, adding 16 bits of reserved space before the length.
> THen defining that when the SETTINGS_HEADER_FRAME_SIZE
> extension setting is received is un-reserves 8 or 16 of those bits
> and the become a longer length value.
>

Agreed. The purpose of the reserved bits was for a future rev of the protocol to move from 24 to 31 bits, *not* for an extension of h2.

(I suppose technically an unrelated extension could steal those bits before we get to h3, unless we explicitly say they are reserved for future extensions of the length field.)

---

From: Rob Trace <[email-masked]>
Date: Fri, 11 Jul 2014 00:53:46 +0000

The other problem with the experiment was that WebSockets was the basis. If the UPGRADE header is removed for any reason (compliant or not) then the WS connection fails.  In HTTP/2, most of the cases where UPGRADE is removed, it will simply result in a HTTP 1.1 connection.  That itself makes the "failure" much different.

Thanks!!

-Rob 

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Wednesday, July 9, 2014 11:45 AM
To: "William Chan (陈智昌)"; [email-masked]
Cc: Roberto Peon; Matthew Kerwin; HTTP Working Group
Subject: Re: Large Frame Proposal

On 2014-07-09 19:15, William Chan (陈智昌) wrote:
> https://insouciant.org/tech/http-slash-2-considerations-and-tradeoffs/
> #Upgrade, including discussions of other deployment options and their 
> success rates.
> ...

It would be interesting to repeat that experiment. It's now 4.5 years later, and deploying Websockets may have caused broken code to be fixed.

Best regards, Julian

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Jul 2014 11:12:10 +1000

Apparently my latest compromise proposal didn't format well for all.... so
here it is again:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    Type (8)   |               Length (24)                     |
 +---------------+-----------------------------------------------+
 |    Flags(8)   |          Stream Identifier (24)               |
 +===============+===============================================+
 |                   Frame Payload (0...)
 +---------------------------------------------------------------+

In summary, we keep the 8 byte header, but reduce the stream ID to 24 bits
and increase the length to 24 bits.

I don't see how we can argue that "we might need" 31 bits of stream ID and
not apply the same "we might need" logic to length.

If 31 bits is too large an effective infinity for length, then it is also
too large an effective infinity for stream ID.

24 bits of each is still 16MB frames and 16M streams in a connection.... I
think both are reasonable guesses for infinity and this is a good
compromise between all the concerns.

cheers

On 11 July 2014 10:53, Rob Trace <[email-masked]> wrote:

> The other problem with the experiment was that WebSockets was the basis.
> If the UPGRADE header is removed for any reason (compliant or not) then the
> WS connection fails.  In HTTP/2, most of the cases where UPGRADE is
> removed, it will simply result in a HTTP 1.1 connection.  That itself makes
> the "failure" much different.
>
> Thanks!!
>
> -Rob
>

---

From: David Krauss <[email-masked]>
Date: Fri, 11 Jul 2014 09:21:45 +0800

On 2014–07–11, at 9:12 AM, Greg Wilkins <[email-masked]> wrote:

> In summary, we keep the 8 byte header, but reduce the stream ID to 24 bits and increase the length to 24 bits.
> 
> I don't see how we can argue that "we might need" 31 bits of stream ID and not apply the same "we might need" logic to length.

Stream IDs are never recycled. Persistent connection longevity is directly proportional to max stream ID.

Length on the other hand depends on hardware evolution. Apples to oranges.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 Jul 2014 18:54:48 -0700

On 10 July 2014 18:12, Greg Wilkins <[email-masked]> wrote:
> think both are reasonable guesses for infinity and this is a good compromise
> between all the concerns.

We've had input to the effect that a rollover would occur too
frequently for some users at a 24-bit identifier.  I think that the
number in question was a rollover every 10 minutes happens anyway.
I'd have to confirm whether that is at 31 or 24.  Given how disruptive
a rollover is, a the extra bits are useful.

Note also that stream identifiers are used elsewhere, so we'd need to
rework things in a few other places if they changed in size.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Jul 2014 12:16:31 +1000

OK, so keep the stream ID at 31 bits.

I cannot see that similar arguments do not apply to length. There are
already use-case for 64KB headers, so a 16b frame size is only just big
enough for them today, so we have zero room for growth and there are some
headers already larger than 64KB, so we are cutting off some users.

I can't live with 16 bit length.

24 bit minimum for me.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Jul 2014 13:11:13 +1000

> I can't live with 16 bit length.

Just to expand why I cannot live with 16 bit lengths.

I can already see valid use-cases today that exceed 16 bit lengths.   We
have decided that 64KB kerberos tickets are acceptable and necessary to
support, so a kerberos authenticated proxy talking to a kerberos
authenticated server will need two such tickets and that will not fit into
a 16bit header block.

So that use-case will just drive fragmentation of headers (continuations by
another name), which breaks the entire purpose of this proposal.

Either we have frames large enough to carry all acceptable headers - or we
need to go back to basics and come up with a framing layer that does not
treat headers specially.

regards

---

From: Roberto Peon <[email-masked]>
Date: Thu, 10 Jul 2014 20:17:30 -0700

I think you're arguing for things at the semantic layer, not he framing
layer-- right now one of the proposals *is* that we treat headers
differently by only allowing them to exist in one frame, which is not how
data is treated.
-=R

On Thu, Jul 10, 2014 at 8:11 PM, Greg Wilkins <[email-masked]> wrote:

> So that use-case will just drive fragmentation of headers (continuations
> by another name), which breaks the entire purpose of this proposal.
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Jul 2014 13:32:16 +1000

> differently by only allowing them to exist in one frame, which is not how

True.  I'm currently advocating for two contradictory positions as I can
see two viable options for headers:

a) They are transported in a dedicated frame type, which must be large
enough to carry all conceivable headers - 24 bit length minimum, but 31bit
better.    This does not move headers to the semantic layer (where they
belong) but is a minimal change to the current draft.  I can live with it.

b) They are transported exactly like data (they are after all just
application data that is not important to framing).  I'd still like to see
a large max frame size, but could probably live with 16 bits even if I
think that would be extremely short sighted.   This is a much bigger change
to the draft and I don't think there is the will to make such a big change
- but I would like to live with it if there were.

What is not viable for me is:

c) Have a multi frame header transport mechanism for headers that is
entirely different to the multi frame transport mechanism for data, with
different rules for flow control and interleaving and expect them to live
side by side without confusion, bugs and QoS issues.  Frame size is
irrelevant in this situation as the solution sucks regardless of frame
size. This is h2-13

cheers

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 10 Jul 2014 23:14:56 -0700

This simply isn't true. We run out of stream IDs more than once a day
between servers.

On Thu, Jul 10, 2014 at 6:12 PM, Greg Wilkins <[email-masked]> wrote:
>> WS connection fails.  In HTTP/2, most of the cases where UPGRADE is removed,
>> it will simply result in a HTTP 1.1 connection.  That itself makes the
>> "failure" much different.
>>
>> Thanks!!
>>
>> -Rob
>>

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 10 Jul 2014 23:16:56 -0700

(sorry -- i'm very late to the response queue this evening)

---

From: Jason Greene <[email-masked]>
Date: Fri, 11 Jul 2014 09:00:29 -0500

So it sounds like the two remaining concerns with the large frame prosal are:

1) Someone might repeat the abuse of SPDY (shove entire files in a frame)

2) Fragmentation.

For [1] I propose a very simple compromise, we use 18 bits + 14 reserved bits. It gives us 256K, which is plenty to fit big headers, and near term growth of communication channels. It also allows big headers without *requiring* a continuation, which was a key element of the spec  The reserved bits make it easy for us to expand this later because all implementations will be trivially compliant in subsequent releases (no wire change necessary). We can of course quibble on the goldilocks number, but that is a small detail compared to the overall benefits of the proposals changes. 

For [2] I think that the use cases are very limited and solving this requires significant revamping of the current continuation form (probably matching some or many of Jeff’s changes). I am not against this, but I think it would be easier to manage the changes if we went forward with the large frame proposal, and then worked out the details of a fragmentation enhancement. Otherwise we might end up in a loop.

Thanks. 

On Jul 10, 2014, at 9:16 PM, Greg Wilkins <[email-masked]> wrote:

> I cannot see that similar arguments do not apply to length. There are already use-case for 64KB headers, so a 16b frame size is only just big enough for them today, so we have zero room for growth and there are some headers already larger than 64KB, so we are cutting off some users.
>

---

From: Roberto Peon <[email-masked]>
Date: Fri, 11 Jul 2014 07:59:55 -0700

As I mentioned before, IIRC we've seen response headers as large as 12mb,
at which point we said: OK, lets have a 2G limit (effectively infinite),
because clearly we can't predict this.

-=R
On Jul 11, 2014 7:05 AM, "Jason Greene" <[email-masked]> wrote:

> For [1] I propose a very simple compromise, we use 18 bits + 14 reserved
> bits. It gives us 256K, which is plenty to fit big headers, and near term
> growth of communication channels. It also allows big headers without
> *requiring* a continuation, which was a key element of the spec  The
> reserved bits make it easy for us to expand this later because all
> implementations will be trivially compliant in subsequent releases (no wire
> change necessary). We can of course quibble on the goldilocks number, but
> that is a small detail compared to the overall benefits of the proposals
> changes.
>
> For [2] I think that the use cases are very limited and solving this
> requires significant revamping of the current continuation form (probably
> matching some or many of Jeff’s changes). I am not against this, but I
> think it would be easier to manage the changes if we went forward with the
> large frame proposal, and then worked out the details of a fragmentation
> enhancement. Otherwise we might end up in a loop.
>
> Thanks.
>
>

---

From: Jason Greene <[email-masked]>
Date: Fri, 11 Jul 2014 10:23:02 -0500

OK. So it sounds like we don’t need to reserve away bits then. We can just stick with the 32 bit limit.

On Jul 11, 2014, at 9:59 AM, Roberto Peon <[email-masked]> wrote:

> As I mentioned before, IIRC we've seen response headers as large as 12mb, at which point we said: OK, lets have a 2G limit (effectively infinite), because clearly we can't predict this.
> 
> -=R
>

---

From: Albert Lunde <[email-masked]>
Date: Fri, 11 Jul 2014 10:57:12 -0500

On 7/11/2014 9:59 AM, Roberto Peon wrote:
> As I mentioned before, IIRC we've seen response headers as large as
> 12mb, at which point we said: OK, lets have a 2G limit (effectively
> infinite), because clearly we can't predict this.

It's reasonable to expect that there are different types of traffic with 
different needs: web browsing, video streaming, and transport of large 
files don't act the same. Active Directory is probably a factor in 
promoting large Kerberos tickets, but seeing them on http/https is a 
corner case outside intranets.

-- 
     Albert Lunde  [email-masked]
                   [email-masked]  (address for personal mail)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 11 Jul 2014 20:15:34 +0000

In message <[email-masked]>

>because clearly we can't predict this.

So there are two questions we need to ask ourselves:

1. Should the protocol support this case ?

2. By default or by configuration ?

3. Who should suffer most ?

My answers are:  Yes, configuration and sender.

Yes, because it is stupid to make a protocol with arbitrary limitations.

Configuration because we should not force all HTTP/2.0 implementations
to over-reserve memory on the off-chance that they ever see one of
these requests.

Sender, because in particular in a case like this, it is important to
give the receiver advance notice that exceptional memory management
will be required.

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 12 Jul 2014 09:27:05 +1000

In summary the case put against this proposal is that some think 31 bits
might be too large.

There has also been a concern put about header fragmentation, but h2-13
cannot be fragmented either, so that is really not a point against this
proposal.

Other than that, I thought we were pretty close to consensus.      None of
the counter proposals made have come close to the near consensus shown in
this thread.

I think we are missing a good opportunity to settle many issues and move on.

On 12 July 2014 06:15, Poul-Henning Kamp <[email-masked]> wrote:

> In message <
> [email-masked]>
> , Roberto Peon writes:
>

---

From: Roberto Peon <[email-masked]>
Date: Fri, 11 Jul 2014 18:23:04 -0700

h2-13 can be fragmented, but not interleaved. Those are different things!
-=R

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 12 Jul 2014 15:03:19 +1200

> h2-13 can be fragmented, but not interleaved. Those are different things!

non-interleaving and fragmenting mean nothing ...

On 12/07/2014 2:59 a.m., Roberto Peon wrote:
> IIRC we've seen response headers as large as 12mb,

... when attempting to manage such large headers. Each status-quo
fragment is only 16KB after all. Easily enough to buffer right?

Ability to know and reject the total up front is crucial. The simplest
and best way to do that is with atomic large frames.

Avoiding the penalty imposition of coping with inifinity on all
implementations we (Greg et al) have the SETTINGS limit.

Amos

> 
> On Fri, Jul 11, 2014 at 4:27 PM, Greg Wilkins wrote:
> 
>>
>> On 12 July 2014 06:15, Poul-Henning Kamp wrote:
>>
>>> In message Roberto Peon writes:
>>>

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 14 Jul 2014 11:28:02 +0000

> It would be interesting to repeat that experiment. It's now 4.5 years later, and deploying Websockets may have caused broken code to be fixed.

true!
it would be really interesting to repeat the experiment for web socket (a lot of proxies and also web servers have been updated to support ws)
and  also run one ad-hoc for http2 as the behaviour between the tow are quite
different at the end

br
Salvatore