Subject: #539: Priority from server to client

From: Mark Nottingham <[email-masked]>
Date: Tue, 1 Jul 2014 17:09:25 +1000

<https://github.com/http2/http2-spec/issues/539>

I thought we'd discussed this a bit in NYC, but don't see anything in the minutes. 

How do people feel about this? Herve has made a proposal in a pull request:
  <https://github.com/http2/http2-spec/pull/526/files>

Personally, my .02 - I get nervous when a protocol element has different semantics depending on what direction it's travelling; e.g., Cache-Control turned out to be very confusing. It feels like that's going on here too; in one direction, it's a priority request, whereas in the other it seems like a declaration...

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Mike Bishop <[email-masked]>
Date: Tue, 1 Jul 2014 14:27:13 +0000

Hervé is correct that the spec currently under-defines what it means when a PRIORITY frame goes from server to client.  However, there are a couple different options for what it might mean:
  1) Server declares the priority it has placed on a stream.
        + What if it's racing with a client-sent PRIORITY frame?
  2) Server informing the client how the client ought to allocate resources
        + Really only makes sense if the client has multiple large PUT/POSTs going on
  3) Server can't send these.

In NYC, we closed a related issue of having the server send its initial priority in the PUSH_PROMISE.  If we picked the first, that would be a natural corollary to the optional presence of PRIORITY in the HEADERS frame.

I do think we should define one, just so there aren't different semantics on both sides of the connection.  I agree that the first makes me itch slightly.

-----Original Message-----
From: Mark Nottingham [mailto:[email-masked]] 
Sent: Tuesday, July 1, 2014 12:09 AM
To: HTTP Working Group
Subject: #539: Priority from server to client

Cheers,

---

From: Johnny Graettinger <[email-masked]>
Date: Tue, 1 Jul 2014 12:22:47 -0400

2) makes sense to me.

I don't think I understand the value of a server advertising to the client
what priority it's using to send a push-stream. By construction, at this
point the client hasn't yet integrated the stream into it's resource
loading flow, and can't make any prioritization decision. As soon as the
client discovers it's need for the resource, that would be an appropriate
time for it to send a PRIORITY to the server anyway.

Also, nothing prevents a client POST from depending on a server push
stream. In this case, a server-sent PRIORITY on that push stream already
has another meaning (it implicitly re-prioritizes the POST stream).

There may be value in relaxing the language in "5.3.5 Default Priorities"
to allow the server to assign an arbitrary initial weight (eg, because it
has knowledge the client doesn't yet have about load priority).

On Tue, Jul 1, 2014 at 10:27 AM, Mike Bishop <[email-masked]>
wrote:

> Hervé is correct that the spec currently under-defines what it means when
> a PRIORITY frame goes from server to client.  However, there are a couple
> different options for what it might mean:
> In NYC, we closed a related issue of having the server send its initial
> priority in the PUSH_PROMISE.  If we picked the first, that would be a
> natural corollary to the optional presence of PRIORITY in the HEADERS frame.
>
> I do think we should define one, just so there aren't different semantics
> on both sides of the connection.  I agree that the first makes me itch
> slightly.
>
> Personally, my .02 - I get nervous when a protocol element has different
> semantics depending on what direction it's travelling; e.g., Cache-Control
> turned out to be very confusing. It feels like that's going on here too; in
> one direction, it's a priority request, whereas in the other it seems like
> a declaration...
>
> Cheers,
>
>
> --

---

From: RUELLAN Herve <[email-masked]>
Date: Tue, 1 Jul 2014 16:37:49 +0000

I'm interested in 1).

When using DASH, a server can push several video fragments to the client. These video fragments should be pushed one after the other in their viewing order. A smart server will do this, and should have a way to inform the client, otherwise the client will need to send re-prioritization to ensure the server does the right thing.

There is a possible race with a client-sent PRIORITY. It could be solved by ruling that the client PRIORITY takes precedence over the server PRIORITY.

Hervé.

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Johnny Graettinger
> Sent: mardi 1 juillet 2014 18:23
> To: Mike Bishop
> Cc: Mark Nottingham; HTTP Working Group
> Subject: Re: #539: Priority from server to client
> 
> 2) makes sense to me.
> 
> I don't think I understand the value of a server advertising to the client what
> priority it's using to send a push-stream. By construction, at this point the client
> hasn't yet integrated the stream into it's resource loading flow, and can't make
> any prioritization decision. As soon as the client discovers it's need for the
> resource, that would be an appropriate time for it to send a PRIORITY to the
> server anyway.
> 
> Also, nothing prevents a client POST from depending on a server push stream.
> In this case, a server-sent PRIORITY on that push stream already has another
> meaning (it implicitly re-prioritizes the POST stream).
> 
> There may be value in relaxing the language in "5.3.5 Default Priorities" to
> allow the server to assign an arbitrary initial weight (eg, because it has
> knowledge the client doesn't yet have about load priority).
> 
> 
> On Tue, Jul 1, 2014 at 10:27 AM, Mike Bishop
> <[email-masked]> wrote:
> 
> 
> 	Hervé is correct that the spec currently under-defines what it means
> when a PRIORITY frame goes from server to client.  However, there are a
> couple different options for what it might mean:
> priority in the PUSH_PROMISE.  If we picked the first, that would be a natural
> corollary to the optional presence of PRIORITY in the HEADERS frame.
> 
> on both sides of the connection.  I agree that the first makes me itch slightly.
> 
> 
> 	Personally, my .02 - I get nervous when a protocol element has
> different semantics depending on what direction it's travelling; e.g., Cache-
> Control turned out to be very confusing. It feels like that's going on here too; in
> one direction, it's a priority request, whereas in the other it seems like a
> declaration...
> 
> 	Cheers,
> 
> 
> 	--

---

From: Johnny Graettinger <[email-masked]>
Date: Tue, 1 Jul 2014 14:01:37 -0400

On Tue, Jul 1, 2014 at 12:37 PM, RUELLAN Herve <[email-masked]>
wrote:

> I'm interested in 1).
>
> When using DASH, a server can push several video fragments to the client.
> These video fragments should be pushed one after the other in their viewing
> order. A smart server will do this,

Makes sense, it seems reasonable to me that a server would want to
initialize push priorities in this way, and assuming that clients abstain
from sending PRIORITY for push-streams they haven't yet discovered a need
for, it sounds like it could provide a benefit. I'm disagreeing only with
the utility of advertisement.

> and should have a way to inform the client, otherwise the client will need
> to send re-prioritization to ensure the server does the right thing.
>

For this to be useful, it pre-supposes a client who has an opinion about
priority, and disagrees with the server's prioritization, yet is willing to
be over-ruled by the server's prioritization anyway. Such a client seems
unlikely to me (how would it reconcile the differing prioriziations?).

Otherwise, we're just talking about saving a PRIORITY frame if the client
happens to find it agrees with the server's advertisements, which doesn't
seem like a required optimization.

There is a possible race with a client-sent PRIORITY. It could be solved by
> ruling that the client PRIORITY takes precedence over the server PRIORITY.
>
> Hervé.
>
> stream.
> natural
> >       I thought we'd discussed this a bit in NYC, but don't see anything
> in the
> > minutes.
> >

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 15 Jul 2014 14:58:45 +1000

We don't seem to be converging on an outcome here.

Herve, do you still want to pursue this? If we take one of the existing options, (2) currently seems most likely.

I think option (4) is to leave it undefined...

On 2 Jul 2014, at 4:01 am, Johnny Graettinger <[email-masked]> wrote:

> On Tue, Jul 1, 2014 at 12:37 PM, RUELLAN Herve <[email-masked]> wrote:
> I'm interested in 1).
> 
> When using DASH, a server can push several video fragments to the client. These video fragments should be pushed one after the other in their viewing order. A smart server will do this,
> 
> Makes sense, it seems reasonable to me that a server would want to initialize push priorities in this way, and assuming that clients abstain from sending PRIORITY for push-streams they haven't yet discovered a need for, it sounds like it could provide a benefit. I'm disagreeing only with the utility of advertisement.
> 
>  
> and should have a way to inform the client, otherwise the client will need to send re-prioritization to ensure the server does the right thing.
> 
> 
> For this to be useful, it pre-supposes a client who has an opinion about priority, and disagrees with the server's prioritization, yet is willing to be over-ruled by the server's prioritization anyway. Such a client seems unlikely to me (how would it reconcile the differing prioriziations?).
> 
> Otherwise, we're just talking about saving a PRIORITY frame if the client happens to find it agrees with the server's advertisements, which doesn't seem like a required optimization.
>  
>

---

From: RUELLAN Herve <[email-masked]>
Date: Tue, 15 Jul 2014 16:41:29 +0000

I still think that it's useful for the server to inform the client on how it intent to order its pushed streams. Amos gave a good example of how it could be used in http://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/0856.html.

At the same time, I understand that using the PRIORITY frame for this may not be the best way to do this, as the semantic of the PRIORITY would not be the same depending on the sender's role.

We could have a twofold outcome:
1. We use option (2) for the meaning of a PRIORITY frame sent from the server to the client.
2. We allow the server to express its priority intentions concerning pushed streams. We could add new optional priority fields to the PUSH_PROMISE frame, that would convey the server intent.

Hervé.

> Sent: mardi 15 juillet 2014 06:59
> To: RUELLAN Herve
> Cc: Mike Bishop; HTTP Working Group; Johnny Graettinger
> > Makes sense, it seems reasonable to me that a server would want to initialize
> push priorities in this way, and assuming that clients abstain from sending
> PRIORITY for push-streams they haven't yet discovered a need for, it sounds
> like it could provide a benefit. I'm disagreeing only with the utility of
> advertisement.
> >
> >
> > and should have a way to inform the client, otherwise the client will need to
> send re-prioritization to ensure the server does the right thing.
> >
> >
> priority, and disagrees with the server's prioritization, yet is willing to be over-
> ruled by the server's prioritization anyway. Such a client seems unlikely to me
> (how would it reconcile the differing prioriziations?).
> >
> happens to find it agrees with the server's advertisements, which doesn't seem
> like a required optimization.
> >
> >

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Jul 2014 10:30:52 -0700

> 2. We allow the server to express its priority intentions concerning pushed streams. We could add new optional priority fields to the PUSH_PROMISE frame, that would convey the server intent.

For a server that is declaring its priority, why can't it just apply
the prioritization it deems best until the client says otherwise?  Is
there some advantage to the server being able to diverge from the
default, but have the client know about the specifics of the
divergence?

---

From: David Krauss <[email-masked]>
Date: Wed, 16 Jul 2014 07:04:34 +0800

> default, but have the client know about the specifics of the

There is no way to prevent servers from diverging from the default, it’s an inevitable application-level optimization.

There is a disadvantage to keeping the initial prioritization secret from the client, because then it does not know when to expect the information. A client could reasonably assume a worst-case scenario. It’s an unnecessary adversarial relationship.

Think of it this way: rather than encourage negotiation, PUSH_PROMISE /PRIORITY lays the server’s cards on the table, to establish a friendly consensus with a client that may not wish to dictate everything.

This applies to option (1) and the text in the pull request (https://github.com/hruellan/http2-spec/commit/bd8052410a5b29a0089f16b721d399105ddee48b ), not to option (2) and server-sent PRIORITY frames. Although I was initially in favor of (2) as well, such behavior seems very application-specific. Upload prioritization commands should be in the ALPN extension domain; this might include PRIORITY frames per se or maybe not.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Jul 2014 16:08:02 -0700

> This applies to option (1) and the text in the pull request (https://github.com/hruellan/http2-spec/commit/bd8052410a5b29a0089f16b721d399105ddee48b ), not to option (2) and server-sent PRIORITY fra…

If you are proposing doing (1), why does this include the
prioritization information in the HEADERS frame that starts the
response and not the PUSH_PROMISE frame?  Surely the latter would make
it easier for a client to provide alternative prioritization in a
timely fashion (and it would be congruent with having the client
provide priority in the request).

---

From: David Krauss <[email-masked]>
Date: Wed, 16 Jul 2014 07:21:04 +0800

On 2014–07–16, at 7:08 AM, Martin Thomson <[email-masked]> wrote:

> response and not the PUSH_PROMISE frame?

There is no HEADERS associated with a push; that functionality is already rolled into PUSH_PROMISE.

Such technical specification seems absent from the pull request anyway. It does not add the flag to the frame, unless more changes exist than the one I linked.

If there’s still any confusion, to be clear: the latest proposal appears to apply only to PUSH_PROMISE and nothing else, so that’s what I’m talking about.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Jul 2014 16:30:22 -0700

> If there’s still any confusion, to be clear: the latest proposal appears to apply only to PUSH_PROMISE and nothing else, so that’s what I’m talking about.

If you are talking about the same proposal (#526), then I don't think
that you claim is well supported by the content of that proposal.  I
think that if that is the case, then we need to have an updated
proposal.

---

From: RUELLAN Herve <[email-masked]>
Date: Wed, 16 Jul 2014 12:20:50 +0000

So I think that where we're heading to is the following:
- The priority information included in a PRIORITY frame or a HEADER frame is a hint from a client to the server on how to allocate resources.
- There are optional priority fields in the PUSH_PROMISE frame that allows a server to expose its intent on how it will allocate resources for pushed streams.

This would suit me.

I can draft an updated proposal for this.

Hervé.

> From: Martin Thomson [mailto:[email-masked]]
> Sent: mercredi 16 juillet 2014 01:30
> To: David Krauss
> Cc: RUELLAN Herve; HTTP Working Group
> > If there’s still any confusion, to be clear: the latest proposal appears to apply
> only to PUSH_PROMISE and nothing else, so that’s what I’m talking about.
>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 16 Jul 2014 10:47:41 -0700

> This would suit me.

I'm just trying to get this straight.  Don't assume that I like it :)
If you can convince more people than David (our chair being the
ultimate person you have to convince), then I'm happy to take a pull
request.

---

From: <[email-masked]>
Date: Wed, 16 Jul 2014 18:47:23 +0000

> ultimate person you have to convince), then I'm happy to take a pull

It sounds reasonable to me.

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jul 2014 11:06:03 -0400

Herve,

I think it might be helpful to have an explicit semantic split in the spec — right now we say “priority” when we really mean *requested* priority.

Talking to Martin about this offline, i think it makes sense to specify our current uses as “requested priority” and add a new field to PUSH_PROMISE as “applied priority”.

Does that make sense?

--
Mark Nottingham   http://www.mnot.net/

---

From: Johnny Graettinger <[email-masked]>
Date: Mon, 21 Jul 2014 11:16:41 -0400

The garden implementation path is that a client sends a PRIORITY frame on a
push stream when it locally identifies a need (and prioritization) for that
resource, and not until then. Thus having the server advertise applied
priority appears useful only to a client which has an opinion about
priority that differs from the server, yet is willing to be overruled by
the server anyway.

Please correct me if I'm wrong, but it's not clear to me that there's a
real use case out there that's enabled by this.

On Mon, Jul 21, 2014 at 11:06 AM, Mark Nottingham <[email-masked]> wrote:

> Herve,
>
> I think it might be helpful to have an explicit semantic split in the spec
> — right now we say “priority” when we really mean *requested* priority.
>
> Talking to Martin about this offline, i think it makes sense to specify
> our current uses as “requested priority” and add a new field to
> PUSH_PROMISE as “applied priority”.
>
> Does that make sense?
>
>
> > - The priority information included in a PRIORITY frame or a HEADER
> frame is a hint from a client to the server on how to allocate resources.
> > - There are optional priority fields in the PUSH_PROMISE frame that
> allows a server to expose its intent on how it will allocate resources for
> pushed streams.
> >
> > This would suit me.
> >

---

From: David Krauss <[email-masked]>
Date: Tue, 22 Jul 2014 13:42:48 +0800

That doesn’t make any sense. The need for a push isn’t identified by a client. The currently foreseen “garden variety” implementation does not interface PUSH_PROMISE streams to the application at all, but only to the user-agent cache. That’s why pushing only works on cacheable resources.

The server identifies the need for the push and it should have the right to send it at whatever rate is appropriate. The client should have the right to know what rate the transfer is supposed to be occurring, for example to know what to down-prioritize when QoS is suffering on a real-time stream from the same server.

On 2014–07–21, at 11:16 PM, Johnny Graettinger <[email-masked]> wrote:

> The garden implementation path is that a client sends a PRIORITY frame on a push stream when it locally identifies a need (and prioritization) for that resource, and not until then. Thus having the server advertise applied priority appears useful only to a client which has an opinion about priority that differs from the server, yet is willing to be overruled by the server anyway.
> 
> Please correct me if I'm wrong, but it's not clear to me that there's a real use case out there that's enabled by this.