Subject: Limiting allowable pre-SETTINGS requests

From: Jason Greene <[email-masked]>
Date: Thu, 5 Jun 2014 11:10:20 -0500

Recent threads have discussed how more limited servers, perhaps running on embedded devices, or resource constrained intermediaries, could use SETTINGS to reduce the decoding overhead both in terms of table size, and depending on the output of #485, potentially usage of huffman.

However, even with this capability, these servers are still required to process requests up until the client has received and processed the SETTINGS frame. I understand that this is purposefully not a full negotiation due to the RTT cost. However, since there is also no form of flow-control on HEADER frames, a client can (and likely will) send as much as it can when the connection is initially created. Depending on the stickiness of traffic patterns (or more precisely lack thereof), this could be a significant volume of traffic.

Should there perhaps be some cooperative limitation on the amount of request data that can be sent before receiving the first SETTINGS frame (i.e a temporary flow-control)? 

An alternative could be for an implementation to ignore the request content and use Retry-After in some scheme to force adoption of the SETTINGS values. However, it becomes guess-work on specifying the time amount.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat

---

From: David Krauss <[email-masked]>
Date: Fri, 6 Jun 2014 06:28:29 +0800

> Recent threads have discussed how more limited servers, perhaps running on embedded devices, or resource constrained intermediaries, could use SETTINGS to reduce the decoding overhead both in terms…

Even embedded servers are heavier than the corresponding clients, and Huffman takes very very little resources — literally a few bytes of RAM.

> However, even with this capability, these servers are still required to process requests up until the client has received and processed the SETTINGS frame. I understand that this is purposefully no…

Sending a “significant volume of traffic” to a “more limited server” will certainly bring it down due to its narrow pipe, regardless of what the installed software does. (Which is, to reject requests outside a very limited scope or causing any problems, and do it early, regardless of what SETTINGS may say is legal.)

> Should there perhaps be some cooperative limitation on the amount of request data that can be sent before receiving the first SETTINGS frame (i.e a temporary flow-control)? 

It sounds like you’re proposing a limited header frame/block size. We don’t have that, regardless of when SETTINGS is applied.

Four kilobytes should be plenty for a proxy to route a stream and relieve the buffering pressure by streaming as HPACK was designed to do, but someone mentioned proxies peeking at cookies too. It seems that we need a closer look at what kind of implementation handles which specific use case. These issues aren’t specific to extra-simple servers.

> An alternative could be for an implementation to ignore the request content and use Retry-After in some scheme to force adoption of the SETTINGS values. However, it becomes guess-work on specifying…

An alternative would be to drop the connection, perhaps with a canned error 431 page.

---

From: Mike Bishop <[email-masked]>
Date: Fri, 6 Jun 2014 01:33:59 +0000

There was, in fact, a draft about this several months back, driven primarily by the fact that it's more painful to reduce flow control window & number of allowed streams than to increase it.  Two different ALPN tokens, with two different initial stream counts; the "constrained" version started at 1 stream and 2K flow control window, while the "full" version started at 100 streams and 64K flow control window.  And of course, regardless of which you negotiated, you can always increase from that without an issue.

See http://tools.ietf.org/html/draft-montenegro-httpbis-http2-server-profiles.  The working group decided not to go this direction, because the driving issue was the pain of reduction, and the general consensus was that reduction wasn't painful enough to warrant the complexity.  If the server is overwhelmed, it can RST every stream it doesn't have the capacity to handle (though it still has to process the HEADERS frames to maintain state).

Ironically enough, I believe the only thing we've retained from that draft are the tokens "h2" and "h2c".

Sent from Windows Mail

From: Jason Greene<mailto:[email-masked]>
Sent: ?Thursday?, ?June? ?5?, ?2014 ?12?:?14? ?PM
To: HTTP Working Group<mailto:[email-masked]>

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 06 Jun 2014 23:14:55 +1200

On 6/06/2014 10:28 a.m., David Krauss wrote:
> 
> Four kilobytes should be plenty for a proxy to route a stream and
> relieve the buffering pressure by streaming as HPACK was designed to
> do, but someone mentioned proxies peeking at cookies too. It seems
> that we need a closer look at what kind of implementation handles
> which specific use case. These issues aren’t specific to extra-simple
> servers.

FWIW the only use-cases I've seen for proxies to peek at Cookie was for
interception proxies to authenticate despite the client-side security
measures, or for load balancers to ensure end-to-end pinning of user
sessions (forcing statefulness on the stateless transfer protocol).
 The WG has decided to ignore interception middleware entirely.
 The Load-balancer use-case is apparently resolved by "just use HTTP/1.1".

Maybe someone has another use case for accessing Cookie but I think the
Load-Balancer case served fine by an HTTP/2 extension between the LB and
the backend servers - provided we are allowed extension frames (or maybe
despite HTTP/2 spec).

Amos

---

From: Jason Greene <[email-masked]>
Date: Fri, 6 Jun 2014 13:47:11 -0500

> Even embedded servers are heavier than the corresponding clients, and Huffman takes very very little resources — literally a few bytes of RAM.

I guess this is moot since that issue was closed. Although, clients can certainly be heavier (e.g. the printer case), and there is a real per token CPU cost. 

> Sending a “significant volume of traffic” to a “more limited server” will certainly bring it down due to its narrow pipe, regardless of what the installed software does. (Which is, to reject reques…

First, that’s an argument against the table size setting in the first place. My point is that the setting doesn’t do it’s purported job when you have lots of short-lived connections pipelining many requests. Second, a resource constrained intermediary need not necessarily mean a narrow pipe. In fact it could the the opposite, a load balancer with a fat pipe, and a ton of traffic to process, and so the bottleneck is CPU and memory.

> It sounds like you’re proposing a limited header frame/block size. We don’t have that, regardless of when SETTINGS is applied.

I was more highlighting the problem and brainstorming potential solutions that don’t involve the RTT hit of a full negotiation which would completely solve it. I wasn’t specifically worried about large headers, but rather clients opening a connection dumping lots of requests on it, and then dropping the connection never actually using the SETTINGS frame. 

Sorry if my explanation was poor.

> Four kilobytes should be plenty for a proxy to route a stream and relieve the buffering pressure by streaming as HPACK was designed to do, but someone mentioned proxies peeking at cookies too. It s…

Due to the delta-compression scheme, streams will always have to be decoded and re-encoded unless the proxy is nothing more than a tcp tunnel. Reverse proxies and load balancers are more advanced than that, and typically need to split traffic by path/host and examine certain headers. So they have to pay that processing cost, with the table size setting being the only way to limit/reduce it. If it was reduced to 0, a proxy could reduce everything down to a scan and copy.
> An alternative would be to drop the connection, perhaps with a canned error 431 page.

The issue with that is that it implies that the request can’t be retried, but really whats desired is that the clients send a request that conforms to the requested SETTINGS.

---

From: Jason Greene <[email-masked]>
Date: Fri, 6 Jun 2014 13:47:11 -0500

---

From: Jason Greene <[email-masked]>
Date: Fri, 6 Jun 2014 14:38:21 -0500

On Jun 6, 2014, at 6:14 AM, Amos Jeffries <[email-masked]> wrote:

> The Load-balancer use-case is apparently resolved by "just use HTTP/1.1”.

Really? I find that perspective surprising. That’s basically saying that any website that needs state and redundancy is better off not using HTTP/2.

Granted I don’t think that situation is all that dire. LBs will likely still support the new protocol even if it is more expensive to process. Of course, if the penalty is too high, it could certainly impact adoption.

> the backend servers - provided we are allowed extension frames (or maybe

That doesn’t really help because it’s the request needs to be pinned, and not aspects of the wire. The client could lose and reestablish a new connection and a new stream, and send another request on the same session, and it needs to go to the right place.