Subject: h2 priority

From: "Roy T. Fielding" <[email-masked]>
Date: Sun, 31 Aug 2014 16:06:47 -0700

This comment is in reference to sections 5.3, 6.2, and 6.3 of

http://tools.ietf.org/id/draft-ietf-httpbis-http2-14.txt

> 5.3.  Stream priority
> 
>    A client can assign a priority for a new stream by including
>    prioritization information in the HEADERS frame (Section 6.2) that
>    opens the stream.  For an existing stream, the PRIORITY frame
>    (Section 6.3) can be used to change the priority.
> 
>    The purpose of prioritization is to allow an endpoint to express how
>    it would prefer its peer allocate resources when managing concurrent
>    streams.  Most importantly, priority can be used to select streams
>    for transmitting frames when there is limited capacity for sending.
> 
>    Streams can be prioritized by marking them as dependent on the
>    completion of other streams (Section 5.3.1).  Each dependency is
>    assigned a relative weight, a number that is used to determine the
>    relative proportion of available resources that are assigned to
>    streams dependent on the same stream.
> 
>    Explicitly setting the priority for a stream is input to a
>    prioritization process.  It does not guarantee any particular
>    processing or transmission order for the stream relative to any other
>    stream.  An endpoint cannot force a peer to process concurrent
>    streams in a particular order using priority.  Expressing priority is
>    therefore only ever a suggestion.
> 
>    Prioritization information can be specified explicitly for streams as
>    they are created using the HEADERS frame, or changed using the
>    PRIORITY frame.  Providing prioritization information is optional, so
>    default values are used if no explicit indicator is provided
>    (Section 5.3.5).

(editorial) The first sentence of last paragraph is a duplicate of the
first paragraph of 5.3.

[... four pages of details on reprioritization based on weighted trees ...]

Does anyone implement this?  I have to wonder, because it appears smack
in the middle of the most important part of the spec and seems to have
no relevance to anything else in the protocol.  The description is very
interesting from an academic perspective (and I like that), but I don't
see how any of this is supposed to work in a live environment where
streams exist for a single request/response. Why bother?

As I described in Paris, waka has a couple bits in every request to
express similar priority as one of four relative choices:
   (speculative, low, HiLo, high)
where speculative is essentially zero priority for things like prefetch
(don't answer until you have done all the other work) and HiLo means
the client needs the rendering metadata right away but the remainder,
as determined by the sender, can be lower priority.

The priority in waka is just expressing how important the response is
to the requester relative to the other requests they have made.

The reason I did not go into any more detail than that is because the
priority information is only a suggestion, like in h2.  Defining how
the recipient is going to reprioritize its stack based on existing
streams is an implementation detail behind the HTTP interface: wishful
thinking and untestable (because it is only a suggestion).  Editorially,
section 5.3 belongs in an appendix, or as a separate spec that has time
to ruminate on the balance between weighted streams and various
response characteristics.

> 6.2.  HEADERS
> 
>    The HEADERS frame (type=0x1) carries name-value pairs.  It is used to
>    open a stream (Section 5.1).  HEADERS frames can be sent on a stream
>    in the "open" or "half closed (remote)" states.
> 
>      0                   1                   2                   3
>      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>     |Pad Length? (8)|
>     +-+-------------+-----------------------------------------------+
>     |E|                 Stream Dependency? (31)                     |
>     +-+-------------+-----------------------------------------------+
>     |  Weight? (8)  |
>     +-+-------------+-----------------------------------------------+
>     |                   Header Block Fragment (*)                 ...
>     +---------------------------------------------------------------+
>     |                           Padding (*)                       ...
>     +---------------------------------------------------------------+
> 
>                            HEADERS Frame Payload

YIKES!  I already mentioned padding in a prior message, so let's just
assume that has been deleted.

Here we have 40 optional bits that might or might not exist as a
preamble to the really important data that everyone needs to see first
in order to do any sort of request routing.  Sorry, that's nuts.

Priority is a suggestion.  Sending it only within a separate frame
type incurs an additional overhead of 64bits, only when it is sent,
assuming we also allow PRIORITY frames to initiate a stream.
I think that is a better trade-off than the current spec.

Sending priority as 2 bits in the frame itself is a better idea, IMO,
but I won't push the WG in that direction.

Sending it as optional bits in front of the headers means that the
most important bits of the protocol are going to be shifting around
in the buffers depending on the value of the PRIORITY bit.  Not good.

If we really want to set priority within a header frame, why don't
we just make it a header field?  A server won't be changing any
priorities until it knows what has been requested, regardless.

Cheers,

Roy T. Fielding                     <http://roy.gbiv.com/>
Senior Principal Scientist, Adobe   <http://www.adobe.com/>

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 1 Sep 2014 10:53:44 +1000

> assume that has been deleted.

and if your END_STREAM proposal is accepted then all we are left with in a
HEADERS frame is

   Header Block Fragment

making this frame identical to CONTINUATION frame.    No need for both.

Further,  if we had restored the END_SEGMENT semantic, then we could use
that instead of the END_HEADERs flag and HEADERS frames would become
identical to DATA frames.

For HTTP semantics, we would simply define (in a layer above framing) that
the first segment is headers, the second data and the optional 3rd is a
trailer.  For websocket semantics, the END_SEGMENT could denote message
boundaries within a stream

We would then have a moderately generic framing layer capable of carry
multiple semantics without the need to revisit the framing layer
implementation for every extension or new protocol.    We would have to
refine hpack a little bit more, but then header compression should never
have been a concern for the framing layer in the first place!

Sorry to keep pushing the same barrow, but I think the current draft has
landed on a local minima of a self consistent design.  Change almost
anything and you pop it out of the local minima and a lot more design
decision will not longer make sense.

cheer

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.