Subject: Server push limited to cache priming?

From: Daniel Sommermann <[email-masked]>
Date: Wed, 19 Mar 2014 09:31:23 -0700

I've noticed that parts of the HTTP/2 spec seem to assume that priming 
the browser cache is the primary purpose of server push. However, it 
seems like server push could be used for other purposes, such as 
delivering async messages from the server to the client. I can imagine 
future application extensions that allow clients to register callbacks 
for when pushed resources arrive. This approach could be much more 
network-efficient than current long-polling techniques. Why does the 
language seem to assume that server push is only used for browser 
caching? E.g. "A server can only push responses that are cacheable". It 
seems unnecessarily restrictive.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 19 Mar 2014 09:56:50 -0700

I think that's a good point. This is an unfortunate part of having perhaps
too (current?) browser-centric a view in some of our discussions.

And this might be a casualty of imprecise terminology. Notably, the
existing text is reasonable for PUSH_PROMISEs, where a push stream is tied
to an "associated" stream. But for server-initiated streams without an
associated stream (where you just initiate with a HEADERS frame), indeed
restricting to only cacheable responses seems unnecessarily restrictive.

Note that in a platform where you might have multiple applications using
the same HTTP/2 connection (e.g. a browser with multiple hosted web apps
multiplexing over the same connection), you need to identify the particular
application to deliver the server-initiated stream to. You can imagine a
URL registration model or something instead, but perhaps the easiest way to
do this for now is to use WebSockets-style message-based communication over
a single HTTP/2 bidirectional stream initiated by the client (so there's an
explicit binding between the client app and the stream).

On Wed, Mar 19, 2014 at 9:31 AM, Daniel Sommermann <[email-masked]> wrote:

> I've noticed that parts of the HTTP/2 spec seem to assume that priming the
> browser cache is the primary purpose of server push. However, it seems like
> server push could be used for other purposes, such as delivering async
> messages from the server to the client. I can imagine future application
> extensions that allow clients to register callbacks for when pushed
> resources arrive. This approach could be much more network-efficient than
> current long-polling techniques. Why does the language seem to assume that
> server push is only used for browser caching? E.g. "A server can only push
> responses that are cacheable". It seems unnecessarily restrictive.
>
>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 19 Mar 2014 09:59:22 -0700

On 19 March 2014 09:31, Daniel Sommermann <[email-masked]> wrote:
> It seems unnecessarily restrictive.

Async messaging is certainly possible with server push, especially
once you have an API extension to support push.  Did you have
something else in mind that requires an uncacheable response?

---

From: Martin Thomson <[email-masked]>
Date: Wed, 19 Mar 2014 10:30:51 -0700

On 19 March 2014 09:56, William Chan (陈智昌) <[email-masked]> wrote:
> This is an unfortunate part of having perhaps too (current?) browser-centric
> a view in some of our discussions.

I don't think that it's that at all.  I know that browsers and browser
use cases dominate the discussion often, but I believe that this
decision was made for another reason.

The reason, as I understand it, relates to the ability of a client to
act upon a response when it is received.  If you push a response that
is not expected and it's not cacheable, then - absent some sort of
eventing API - it has to be discarded.  This is more a symptom of
wanting to minimize the delta for using applications, and a concern
that we don't completely understand all the implications of a choice
to allow non-cacheable responses.  Once we have eventing APIs, then
perhaps we can lift the restriction, but it's harder to remove
functionality if we made a mistake.

---

From: Daniel Sommermann <[email-masked]>
Date: Wed, 19 Mar 2014 11:11:12 -0700

On 03/19/2014 09:59 AM, Martin Thomson wrote:
> Did you have

Not in particular, but the language of the spec assumes that the purpose 
of server push is cache priming. Before I run out and implement a 
notification channel or other non-caching features on top of server 
push, it would be good to know that this is not an "unexpected" or 
"unsupported" use-case.

On 03/19/2014 09:56 AM, William Chan (陈智昌) wrote:
> Note that in a platform where you might have multiple applications 
> using the same HTTP/2 connection (e.g. a browser with multiple hosted 
> web apps multiplexing over the same connection), you need to identify 
> the particular application to deliver the server-initiated stream to. 
> You can imagine a URL registration model or something instead, but 
> perhaps the easiest way to do this for now is to use WebSockets-style 
> message-based communication over a single HTTP/2 bidirectional stream 
> initiated by the client (so there's an explicit binding between the 
> client app and the stream).

I'm imagining a scheme something like this:

-> GET /subscribe?chan=my_chan (stream=1)

<- PUSH_PROMISE (assoc-stream=1, promised-stream=2)
<- HEADERS and DATA (stream=2)

<- PUSH_PROMISE (assoc-stream=1, promised-stream=4)
<- HEADERS and DATA (stream=4)

...etc

On the same connection, the client could be subscribed to the same 
channel multiple times, which is probably just a client bug and would 
lead to duplicate notifications. I don't see how this is fatal however. 
The server or intermediary doesn't have to do any extra bookkeeping (I 
think you were getting at this approach with your last point). It would 
just forward based on the associated stream. You are right that we don't 
really need both a PUSH_PROMISE and HEADERS in this case. In some ways 
SPDY's push mechanism is more appropriate for this use case than HTTP/2.

On 03/19/2014 10:30 AM, Martin Thomson wrote:
> Once we have eventing APIs, then perhaps we can lift the restriction, 
> but it's harder to remove functionality if we made a mistake. 
Understandable. I guess I'd like to get us thinking about those eventing 
APIs sooner than later to make sure the caching restrictions don't bite 
future implementations. If there are opportunities to safely broaden the 
language of the spec to acknowledge other uses for server push, I think 
we should make those changes.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 20 Mar 2014 11:51:40 +1100

Speaking personally -- 

I'm going to continue to push back on non-caching uses of server push in HTTP/2; our charter explicitly tells us to be semantically backwards-compatible with HTTP/1, and that means no new semantics. The only way to fit server push into HTTP's existing semantics is to consider it a cache update.

In other words, if you overload server push to mean "async message" and use a new client API to get them, that's not a HTTP API. It'd also necessitate figuring out how to distinguish those updates from pushes that *are* intended for the cache.

OTOH it should be possible to layer async messaging over caching in a way that takes advantage of server push, somewhat like COMET does. I've started to sketch out one mechanism that would contribute to that here:
  http://tools.ietf.org/html/draft-nottingham-http-patch-status-00#appendix-A

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 20 Mar 2014 01:07:36 +0000

I don't think backward compatibilty means nothing new can be allowed.  
It just means everything old must still be supported.  If a pathway 
doesn't support all new features (e.g. a 1.1 gateway in the chain), then 
that is a different issue, and some new features may not be available in 
that deployment case, but shouldn't be prohibited in the spec.

Cheers

Adrien

------ Original Message ------
From: "Mark Nottingham" <[email-masked]>
To: "Martin Thomson" <[email-masked]>
Cc: "William Chan (陈智昌)" <[email-masked]>; "Daniel Sommermann" 
<[email-masked]>; "HTTP Working Group" <[email-masked]>
Sent: 20/03/2014 1:51:40 p.m.
Subject: Re: Server push limited to cache priming?

>Speaking personally --
>
>I'm going to continue to push back on non-caching uses of server push 
>in HTTP/2; our charter explicitly tells us to be semantically 
>backwards-compatible with HTTP/1, and that means no new semantics. The 
>only way to fit server push into HTTP's existing semantics is to 
>consider it a cache update.
>
>In other words, if you overload server push to mean "async message" and 
>use a new client API to get them, that's not a HTTP API. It'd also 
>necessitate figuring out how to distinguish those updates from pushes 
>that *are* intended for the cache.
>
>OTOH it should be possible to layer async messaging over caching in a 
>way that takes advantage of server push, somewhat like COMET does. I've 
>started to sketch out one mechanism that would contribute to that here:
>

---

From: Roberto Peon <[email-masked]>
Date: Wed, 19 Mar 2014 18:10:48 -0700

I'm not too worried about this.
The types of things that would really use server-initiated streams but not
push something into a cache are the kinds of things best served with the
WebSocket API or something similar.
-=R

On Wed, Mar 19, 2014 at 6:07 PM, Adrien de Croy <[email-masked]> wrote:

>
> I don't think backward compatibilty means nothing new can be allowed.  It
> just means everything old must still be supported.  If a pathway doesn't
> support all new features (e.g. a 1.1 gateway in the chain), then that is a
> different issue, and some new features may not be available in that
> deployment case, but shouldn't be prohibited in the spec.
>
> Cheers
>
> Adrien
>
>
> Cc: "William Chan (陈智昌)" <[email-masked]>; "Daniel Sommermann" <
> [email-masked]>; "HTTP Working Group" <[email-masked]>
>> I'm going to continue to push back on non-caching uses of server push in
>> HTTP/2; our charter explicitly tells us to be semantically
>> backwards-compatible with HTTP/1, and that means no new semantics. The only
>> way to fit server push into HTTP's existing semantics is to consider it a
>> cache update.
>>
>> necessitate figuring out how to distinguish those updates from pushes that
>> *are* intended for the cache.
>>
>> OTOH it should be possible to layer async messaging over caching in a way
>> that takes advantage of server push, somewhat like COMET does. I've started
>> to sketch out one mechanism that would contribute to that here:

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 20 Mar 2014 12:27:42 +1100

... and I *think* that's where we're at with the current text. I just don't want to go explicitly documenting things that we're not designing / building into the protocol.

On 20 Mar 2014, at 12:07 pm, Adrien de Croy <[email-masked]> wrote:

> 
> I don't think backward compatibilty means nothing new can be allowed.  It just means everything old must still be supported.  If a pathway doesn't support all new features (e.g. a 1.1 gateway in the chain), then that is a different issue, and some new features may not be available in that deployment case, but shouldn't be prohibited in the spec.
> 
> Cheers
> 
> Adrien
> 
> Cc: "William Chan (陈智昌)" <[email-masked]>; "Daniel Sommermann" <[email-masked]>; "HTTP Working Group" <[email-masked]>

---

From: David Krauss <[email-masked]>
Date: Fri, 21 Mar 2014 07:49:29 +0800

On 2014–03–20, at 1:30 AM, Martin Thomson <[email-masked]> wrote:

> eventing API - it has to be discarded.  

It sounds like it would be useful to have a header to specify that a resource should be evicted from the cache once retrieved.

Safe to ignore, but if used it would avoid a form of cache pollution that currently does not exist.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Mar 2014 10:59:50 +1100

That's interesting. Probably just a Cache-Control directive, e.g.,

  Cache-Control: max-age=60, once

Like you say, just a hint.

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 21 Mar 2014 18:50:39 +1300

> Like you say, just a hint. 

I dont get why there is a need to involve new protocol extensions at all
for this use case.

A server which wishes the client to get it immediately and avoid caching
is already free to send Cache-Control:no-store or max-age=0 to prevent
intermediaries caching it regardless of what happend to the PUSH.

A Server which wishes the client to be able to pull it from cache once
can use Cache-Control:private,must-revalidate so no other client gets it
and a no-store and/or Expires can be attached on the revalidation reply
to purge it from cache.

Intermediaries which support per-client caching can make use of the
private to store it and respond to a delayed GET as a revalidated
private object.

If an intermediary is not going to relay the PUSH on to the client and
not to cache it per-client at all then there is no way for the use-case
to work regardless of protocol specs.

So where does the new protocol semantics or extensions come in?
It may not work today due to use-cases not being implemented for yet but
that is not a protocol issue AFAICT.

Amos

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 24 Mar 2014 18:38:17 +0900

> WebSocket API or something similar.

Hello Roberto,

I agree that currently, these kinds of things are best served with the 
WebSocket API. But using the WebSocket API means throwing away 
addressability (linking, bookmarking, searchability,...) and caching.

There are applications that can benefit both from a true push channel 
(e.g. the WebSocket server->client direction or HTTP2 PUSH) and from 
addressability and caching.

And combining these may have other benefits, such as greater ease of 
reconnecting after the connection is broken. WebSocket currently leaves 
this completely up to the application.

Regards,   Martin.