Subject: HTTP2 server-side stream creation

From: Fedor Indutny <[email-masked]>
Date: Tue, 07 Jul 2015 22:43:37 +0000

Hello!

During development of [spdy-transport][0] library for node.js, one
of the contributors asked about the server-side stream creation in
HTTP2 protocol.

Obviously, the most straightforward way is to do a PUSH_PROMISE on
existing client-initiated stream, but it appears to me that the
server-initiated streams created using HEADERS frame are valid
too.

Is there any place in spec that prohibits it?

Thank you very much for your time,
Fedor.

[0]: https://github.com/indutny/spdy-transport <https://github.com/indutny/spdy-transport>
[1]: https://github.com/indutny/spdy-transport/issues/6 <https://github.com/indutny/spdy-transport/issues/6>

---

From: Cory Benfield <[email-masked]>
Date: Wed, 8 Jul 2015 08:49:46 +0100

>From section 8.1 of RFC 7540[0]:

> A client sends an HTTP request on a new stream, using a previously
> unused stream identifier (Section 5.1.1).  A server sends an HTTP
> response on the same stream as the request.

My reading is that this forbids a 'server' from sending a HEADERS
frame first, because servers send responses on already-opened streams.

You could pretty easily construct a semantic for this that essentially
turns HTTP/2 into a peer-to-peer communication stream, with both sides
of the connection being free to issue requests. This could plausibly
be very valuable in systems that use HTTP/2 as an RPC transport. I
suspect most clients will currently not allow that behaviour, however,
so if you wanted it it might be best to propose it as a negotiated
HTTP/2 extension, per section 5.5 of RFC 7540[1]. If you (or anyone
else on the list) think this is interesting I'd be happy to co-author
a draft to propose it.

Cory

[0]: https://tools.ietf.org/html/rfc7540#section-8.1
[1]: https://tools.ietf.org/html/rfc7540#section-5.5

---

From: Fedor Indutny <[email-masked]>
Date: Wed, 08 Jul 2015 08:18:49 +0000

+David Dias

Thank you very much for a detailed response, Cory!

I think it might be very useful for communication with docker
containers, and as far as I know it was one of the applications of
SPDY protocol for David.

David,

Maybe you have something to add here?

Thank you!

On Wed, Jul 8, 2015 at 12:49 AM, Cory Benfield <[email-masked] <mailto:[email-masked]>> wrote:

Cory

[0]: https://tools.ietf.org/html/rfc7540#section-8.1 <https://tools.ietf.org/html/rfc7540#section-8.1>
[1]: https://tools.ietf.org/html/rfc7540#section-5.5 <https://tools.ietf.org/html/rfc7540#section-5.5>

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 08 Jul 2015 22:30:11 +1200

On 8/07/2015 7:49 p.m., Cory Benfield wrote:
> On 7 July 2015 at 23:43, Fedor Indutny wrote:

Technically.

However, HTTP/1.x currently still exists in the world. A surprisingly
large number of connections one way or another pass over at least one
HTTP/1.x transit hop. So for now any implementations will have to cope
with translation to HTTP/1.x where server requests are not possible.

HTTP/2 was designed with that in mind, thus it does not define
server-initiated semantics. But also does not forbid them outright,
since extensions or HTTP/3 may one day have a need to define it.

If you have a strong use case for it I suggest writing up a SETTINGS
extension which can be negotiated between two endpoints. That way 2->1.1
gateway devices can negotiate its absence for the 1.1 hops and things
still work.

HTH
Amos

---

From: Cory Benfield <[email-masked]>
Date: Wed, 8 Jul 2015 11:58:17 +0100

> gateway devices can negotiate its absence for the 1.1 hops and things

I can certainly see a case for it. I'll see if I can whip up an ID
sometime today to use as a starting point for discussion on this.

Cory

---

From: Mike Bishop <[email-masked]>
Date: Wed, 8 Jul 2015 17:50:27 +0000

Alternately stated, HTTP/2 defines two "things" -- a multiplexed binary framing layer, and a mapping of HTTP semantics to that framing layer.

The framing layer doesn't prohibit server-initiated streams, but outside of Server Push the mapping of HTTP semantics doesn't use them.  A server-initiated stream wouldn't mean anything to an HTTP client, until you have an extension that defines them.

-----Original Message-----
From: Amos Jeffries [mailto:[email-masked]] 
Sent: Wednesday, July 8, 2015 3:30 AM
To: [email-masked]
Subject: Re: HTTP2 server-side stream creation

On 8/07/2015 7:49 p.m., Cory Benfield wrote:
>> server-initiated streams created using HEADERS frame are valid too.
> 

Technically.

However, HTTP/1.x currently still exists in the world. A surprisingly large number of connections one way or another pass over at least one HTTP/1.x transit hop. So for now any implementations will have to cope with translation to HTTP/1.x where server requests are not possible.

HTTP/2 was designed with that in mind, thus it does not define server-initiated semantics. But also does not forbid them outright, since extensions or HTTP/3 may one day have a need to define it.

If you have a strong use case for it I suggest writing up a SETTINGS extension which can be negotiated between two endpoints. That way 2->1.1 gateway devices can negotiate its absence for the 1.1 hops and things still work.

HTH
Amos

---

From: Cory Benfield <[email-masked]>
Date: Wed, 8 Jul 2015 19:30:32 +0100

With the IETF Prague meeting so close I currently cannot submit any internet drafts, but in the meantime a quick top-level proposal is readable here[0] if anyone would like to provide feedback.

I’ll submit the ID properly when submissions re-open on the 19th.

Cory

[0]: https://raw.githubusercontent.com/Lukasa/http2-p2p/master/draft-benfield-http2-p2p-00.txt

> On 8 Jul 2015, at 18:50, Mike Bishop <[email-masked]> wrote:
>

---

From: Ilari Liusvaara <[email-masked]>
Date: Wed, 8 Jul 2015 22:03:02 +0300

> [0]: https://raw.githubusercontent.com/Lukasa/http2-p2p/master/draft-benfield-http2-p2p-00.txt

"This limits some of the restrictions of RFC 7540  [RFC7540] Section 8."

s/limits/lifts/  (dunno)?

Also, I think that for bidirectional push to work, one also needs lift
the restriction that server can't set push enabled flag (RFC7540,
section 8.2, last paragraph).

-Ilari

---

From: Cory Benfield <[email-masked]>
Date: Wed, 8 Jul 2015 21:13:18 +0100

> On 8 Jul 2015, at 20:03, Ilari Liusvaara <[email-masked]> wrote:
> 

Great spot, thanks.

> section 8.2, last paragraph).

I think you’re right, but I think this is symptomatic of a broader point, which is that any setting that is dependent on the idea of a ‘client’ or a ‘server’ needs to be rethought. In the case of RFC 7540 I think you’re right that this is only SETTINGS_ENABLE_PUSH, but a note should be made for the case where this extension is combined with others.

I’ve applied some draft updates in response to this feedback. Thanks!

Cory

---

From: David Dias <[email-masked]>
Date: Wed, 08 Jul 2015 19:58:31 +0000

(Replying from my personal email)

Thank you for adding me to the thread. We’ve and seen and worked with several implementations of the SPDY framing layer recently, most notable are muxado[1], yamux[2] and the one we currently use on the IPFS project, spdystream[3]. One of the main reasons behind these efforts is due to the fact that SPDY offers a spec for a stream multiplexing with back pressure and flow control, including priority mechanisms, which makes it extremely interesting  for cross compatibility between several implementations in different languages.

Currently, with have cross compatibility with spdystream (Docker’s Go implementation) and spdy-transport (Fedor’s Node.js implementation) working, both respecting the SPDY framing layer specification, which suits the purpose and as long as the listener is also able to ‘request’ a new stream to open from the ‘dialer', of course, other than a push stream. SPDY, or the HTTP2 framing layer, have a great use case for applications that need a great amount and several types of data to flow.

Right now this solutions exist due to the understanding that the SPDY framing layer can be mounted as a stand alone stream muxer, but if there is a way to make it official that there is a great use case and benefit of having it as its own extension to the protocol, we, Protocol Labs (http://ipn.io/ <http://ipn.io/>), would love to contribute.

Thank you, 

[1] - https://github.com/inconshreveable/muxado <https://github.com/inconshreveable/muxado>
[2] - https://github.com/hashicorp/yamux <https://github.com/hashicorp/yamux>
[3] - https://github.com/docker/spdystream <https://github.com/docker/spdystream>

David Dias
@daviddias <https://twitter.com/daviddias>

> ----------- Forwarded message -----------
> From: [email-masked] <mailto:[email-masked]>
> Date: Wed Jul 08 2015 01:18:32 GMT-0700 (PDT)
> To: [email-masked] <mailto:[email-masked]>
> Cc: [email-masked] <mailto:[email-masked]>, [email-masked] <mailto:[email-masked]>
> 
> 
>> +David Dias
>>