Subject: Overhead of HTTP/2 Stream Management.

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 11:08:50 -0700

I'm in the process of implementing HTTP/2 into Avuna, and I have a question.
>From a performance point of view, why do streams maintain a open/close
state? There seems to be a considerable overhead for it, yet it doesn't
seem necessary. Think of packet IDs in DNS, it just returns with the same
ID to say it is a response to a specific request. What is the benefit of a
detailed stream-like system, versus a simple request/response identifier?

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 5 Apr 2015 20:36:51 +0200

> detailed stream-like system, versus a simple request/response identifier?

HTTP is a bit more complete than a simple request/response model. For
example when sending a body accompanying a POST, you need to reference
the request the data belong to. Also the push mechanism requires to
associate the pushed requests/responses with an existing request, so
it makes sense to use IDs there as well.

In general, since the streams are multiplexed, I hardly see how you
could get rid of stream IDs. I think you were thinking about the existing
pipeline mode of HTTP/1 which forces all responses to be delivered in the
same order as the requests and which suffers from massive head-of-line
blocking.

With that said, I agree that H/2 is not easy to implement, but for now
my experience is that re-architecturing existing programs to support
multiple streams over a single connection is probably much more
difficult than implementing the protocol itself, though this will
obviously depend on the programs of course.

Regards,
Willy

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 11:45:53 -0700

My thoughts is that you just don't use so much overhead. You don't get rid
of stream IDs, you just don't need so much complex things surrounding it.
Example: You append a header to HTTP/1.1 request, with a response ID,
server responds with it. Server can push responses by sending a unsent ID &
request path in a header.
You compress headers with HPACK & allow a chunked-like system which allows
a response over many/unspecified number of packets, but also specifying a
'stream ID'.
Such a system could be engineered to be cross-compatible with HTTP/1.1(via
an upgrade system not unlike HTTP/2), save considerable overhead(and time
re-implementing the entire networking system of current applications), and
lose no capability.
Perhaps I'm not getting something, but I don't see the issue with the idea
yet.

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 5 Apr 2015 20:57:55 +0200

> request path in a header.

You still need the stream IDs in the frames themselves so that you know
which stream each frame belongs to.

Multiplexed systems always look simple at first, until you start to
implement them, cover the corner cases (eg: who closes first etc) and
you finally realize once everything is done how much your system looks
like tcp...

There was an elegant (in my opinion) simplification in H/2 compared to
other systems, the stream IDs are always incremented until the largest
encodable ID is reached, which is where a new connection must be used.
I find this elegant because you don't need to keep track of IDs in use
vs available ones and it really simplifies a number of things (eg: no
risk to have late frames from an old stream using the same ID).

It doesn't please me either to have to implement such a complex system
but I am absolutely convinced that it can hardly be simplified further
as long as we want non-blocking, multiplexed streams. I have already
implemented multiplexed streams in the past for some projects, and it
resulted in almost the same design (but more complex).

Willy

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 13:08:28 -0700

The way HTTP works though, we don't need streams in such a conventional and
TCP-like way. We only need multiplexed packets to carry data over, so just
associate request/response pairs with an ID, and allow server push via
server sending the request path in a header too. Why do we even need a
frame structure? It's unnecessary overhead. Same with the virtual streams.

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 6 Apr 2015 09:15:04 +1000

Max,

I don't see much difference between the stream overheads of HTTP/2 vs the
connection overheads of HTTP/1.

Both need open/close state kept and even in HTTP/1 that state is moderately
complex as you can be half closed in and/or half closed out; the response
can complete before the request; there are multiple sources of events
(application vs network) that can race on state changes; the server has a
requirement to reliably deliver a serialised event stream to the
application without duplicates or loopbacks.    Unless the server/client
keeps good atomic state on the open/closed status, then there are going to
be lost events and/or leaked resources.

In jetty the vast majority of this state overhead is in common code used by
both HTTP/1 and HTTP/2.    This code used to be a lot simpler in older
versions of our HTTP/1 on server... but it was wrong code that missed many
edge cases when presented with fully asynchronous applications.   Closing
asynchronous streams is just complex and multiplexing changes that very
little. It just makes the network event stream look a little different and
some events are distributed to multiple listeners.

cheers

On 6 April 2015 at 06:08, Max Bruce <[email-masked]> wrote:

> The way HTTP works though, we don't need streams in such a conventional
> and TCP-like way. We only need multiplexed packets to carry data over, so
> just associate request/response pairs with an ID, and allow server push via

-- 
Greg Wilkins <[email-masked]>  @  Webtide - *an Intalio subsidiary*
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 16:37:20 -0700

Well, I proposed a HTTP/1.1 addition to Mark, that prevents HOL blocking &
allows server pushing in HTTP/1.1, with 100% backwards compatibility, and
relative ease to implement, and can even start HPACK without any direct
protocol negotiations. TCP is in charge of ensuring that content gets where
it needs to go, if that TCP is untimely closed, that's something the client
just has to re-request. As for the connection/stream overhead, you can have
only one case of initializing TCP, and zero stream overhead, using 1-2 HTTP
headers, that allow server pushing and prevent HOL blocking if implemented
on both client and server(and if not, prepare the responses for the client
before they ask).

On Sun, Apr 5, 2015 at 4:15 PM, Greg Wilkins <[email-masked]> wrote:

> Both need open/close state kept and even in HTTP/1 that state is
> moderately complex as you can be half closed in and/or half closed out; the
> response can complete before the request; there are multiple sources of
> events (application vs network) that can race on state changes; the server
> has a requirement to reliably deliver a serialised event stream to the
> In jetty the vast majority of this state overhead is in common code used
> by both HTTP/1 and HTTP/2.    This code used to be a lot simpler in older

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 6 Apr 2015 01:49:11 +0200

On Sun, Apr 05, 2015 at 01:08:28PM -0700, Max Bruce wrote:
> server sending the request path in a header too.

Would you please describe what you call "packets" here and how you associate
them with a request/response pair, and when a request/response message has
begun, when it is finished, when it is certain that a "packet" from an
aborted response is to be ignored, etc ? I think you'll come to a solution
pretty similar to what was obtained here in the end.

Willy

---

From: Roberto Peon <[email-masked]>
Date: Sun, 5 Apr 2015 16:50:49 -0700

If that is true, then you may want to write up a server and client
implementation, test/deploy it in the real world, and the write up.submit a
draft.
-=R

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 16:55:04 -0700

Each request adds a X-Req-ID to the headers as such:

GET / HTTP/1.1
X-Req-ID: random-unique-per-connection-id
other headers...

and the server responds...

HTTP/1.1 200 OK
X-Req-ID: request-id-here
X-Req-Target: /
other headers...

BUT, can also respond twice or more for server pushing:

HTTP/1.1 200 OK
X-Req-ID: -PUSH-
X-Req-Target: /logo.png
other headers...

The best part? If the server doesn't support it, it just responds as
normal, and the client interprets as normal.
I have a live implementation here: http://www.avuna.org/ OR
https://github.com/JavaProphet/Avuna-HTTPD

^^^ Excerpt from my email to Mark.

Once you have verified on both ends that they support this extension based
on the response, you can decide how to pair response->request, and the
request(which in my server is just a duplication of the original, removing
the ETag). Pushed resources may already be cached, as determined by the
server.
You can from the on also use HPACK if supported.

On Sun, Apr 5, 2015 at 4:49 PM, Willy Tarreau <[email-masked]> wrote:

> and

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 16:56:39 -0700

I wrote a server implementation fully except no HPACK yet(working on now).
As for a client impl, I tested using a header modifier plugin for Firefox,
but for a full on browser/extension would be out of my skill set. As for
submitting a draft, I emailed Mark, and don't know how to submit one myself.

On Sun, Apr 5, 2015 at 4:50 PM, Roberto Peon <[email-masked]> wrote:

>> Well, I proposed a HTTP/1.1 addition to Mark, that prevents HOL blocking
>> & allows server pushing in HTTP/1.1, with 100% backwards compatibility, and
>>> I don't see much difference between the stream overheads of HTTP/2 vs
>>> the connection overheads of HTTP/1.
>>>

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 6 Apr 2015 02:16:16 +0200

> BUT, can also respond twice or more for server pushing:

But that doesn't work. You don't have multiplexed streams this way, just
full requests then full responses. Try to download two videos in parallel
and you'll have a full video and once completely downloaded, you'll get
the second one. What you are proposing here is just a very minor add-on
to what we were doing with HTTP/1, basically you allow the server to
respond out of order, that's all.

H2 supports multiple streams over the same TCP connection to limit the
buffer bloat issue, to reduce the connection counts, and to try to
conserve the TCP congestion windows. And it supports these streams
in *parallel*, yours are serialized.

Regards,
Willy

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 17:28:37 -0700

I get your point, but using something like chunked transfer encoding, you
could use the same idea of id-assigned packets to have intermittent
requests.
However, generally your not performing requests while streaming, but
support would be for the best, and I'll definitely set to do this. I'm
thinking we have response-like chunked encoding for each chunk. ex.

request for video

chunk 1 =
CHK * HTTP/1.1
X-Req-ID: -theid/orpush-

FFFF
65535 bytes of stream.

---

From: Max Bruce <[email-masked]>
Date: Sun, 5 Apr 2015 17:35:44 -0700

Sorry, I formatted it as a request, this would be more proper:

HTTP/1.1 206 Partial Content (or some custom status code if you want to be
original)
X-Req-ID: blah
(other headers MAY be necessary, probably not, original response will
likely have anything)

FFFF
65535 bytes of data

---

From: "Walter H." <[email-masked]>
Date: Wed, 8 Apr 2015 09:52:42 +0200

> in *parallel*, yours are serialized.

with the side effect or disadvantage in comparison to HTTP/1.x, that two
separate TCP connection need not be neccessarily routed identical;

now think of a problem, that one of the two TCP connections gets closed;
the other one remains open and only one download gets broken;
in case of HTTP/2 this would be with all transmissions within the same TCP
transmission -> single-point-of-failure;

by the way, the samples were with HTTP/1.x and HTTP/2