Subject: Re: h2 priority

From: Cory Benfield <[email-masked]>
Date: Mon, 1 Sep 2014 08:46:50 +0100

On 1 September 2014 01:53, Greg Wilkins <[email-masked]> wrote:
> Sorry to keep pushing the same barrow, but I think the current draft has
> landed on a local minima of a self consistent design.  Change almost
> anything and you pop it out of the local minima and a lot more design
> decision will not longer make sense.

Thank you! I've been feeling weird about Roy's suggestions for a
little while but couldn't articulate what I was thinking, but this is
it. The WG has been over many of these issues already and had made
small changes, but had rejected the larger changes (in part out of a
concern for already-existing implementations, and in part out of a
desire to get HTTP/2 out the door).

I think that if we make any of the (relatively substantial) changes
Roy is suggesting we'll want to go back and look really hard at the
division between the framing layer and the semantic layer. However, I
anticipate substantial push-back on making these changes. The WG has
been focused on moving quickly, and regardless of how complex the
specification is we do have implementations that can interop. In
principle HTTP/2 *could* be pushed out the door right now and the
world would not end.

However, the WG should decide what it means to do this correctly. We
either slow HTTP/2 down to refine it or we accept that almost
immediately after getting it out the door we'll need to start looking
at 'fixing' it for HTTP/3. There is clearly a crowd in favour of doing
either of those things in this WG, but the browser implementations are
noticeably absent from that crowd. They are the group that need
convincing.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 01 Sep 2014 07:57:43 +0000

--------
In message <[email-masked]>

>division between the framing layer and the semantic layer.

Yes please!

>In principle HTTP/2 *could* be pushed out the door right now and the
>world would not end.

... nor would it become a dramatically better place, or for that
matter, a worse place if HTTP/2 only happened in one or two years
time.

>at 'fixing' it for HTTP/3.

The fundamental problem is that against the sage advice from Frederick
P. Brooks, the WG has taken the prototype and polished it a bit with
the intent to ship it to customers on a rushed schedule.

Given that we are working on a replacement for the worlds most
popular protocol, that is not rational behaviour.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Simone Bordet <[email-masked]>
Date: Tue, 2 Sep 2014 15:22:42 +0200

Hi,

> popular protocol, that is not rational behaviour.

My impression as well.

-- 
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 2 Sep 2014 16:48:33 +0300

> division between the framing layer and the semantic layer.

What I think is one way to change that division (going a bit further
than Roy's suggestions):

Frame header:
- Frame format changed to (stream(32)+type(8)+length(24)).
- Frame types with always zero / always nonzero stream ID can share
  a frame type[1][2].

Header blocks:
- The whole header block is sent at once (no CONTINUATION). Header
  blocks are extempt from frame size limits[3][4].
- If header block creates a stream, :scheme has to be first[5].
- Unknown :scheme (in stream creation) triggers stream error[6].
- All header names and values are 8-bit clean. No ordering
  restrictions except one noted above.
- Schemes can place limits on valid header names, values and
  ordering (stream error to violate).

Flags:
- END_STREAM is replaced by END_STREAM frame.
- PADDED is replaced by PADDING frame.
- END_HEADERS is removed (no CONTINUATION).
- PRIORITY is removed (use PRIORITY frame).
- ACK is replaced by SETTINGS_ACK / PONG frames.

Frame sequencing:
- Stream frame sequencing at framing level is only constrained by the
  stream state machine.
- Schemes may have further restrictions (stream error to violate)[7].

Push:
- SETTINGS_ENABLE_PUSH can be set to 1 in both directions (HTTP
  will reset stream if you try to push client-to-server).

This removes most HTTP-specific stuff from the framing layer (except
for SETTINGS space and error codes space)[8].

[1] Vast majority of frames would only be valid for one of those and
one would have to check anyway.

[2] RST_STREAM/GOAWAY and WINDOW_UPDATE should have the same numbers.

[3] CONTINUATION already busts frame size limits and interactivity just
as bad.

[4] Incentive to not send excessive headers, 16MB is darn near infinite
for present purpose.

[5] For header streaming to work, it has to be first.

[6] Responding at HTTP level no longer works.

[7] Things like not starting response with a DATA frame in HTTP.

[8] CONNECT_ERROR, description of SETTINGS_MAX_HEADER_LIST_SIZE.

-Ilari

---

From: Patrick McManus <[email-masked]>
Date: Tue, 2 Sep 2014 11:25:38 -0400

> the intent to ship it to customers on a rushed schedule.

I disagree that 2.5 years in committee is rushed by any definition. Some
folks can't be bothered to generate running code or sometimes even comment
for much of that time; but that pattern persists in any group no matter the
term or the people (I too am guilty at times.)

I'm also quite confident that more time in committee will not make this any
better. Heck, the last iteration literally rearranged the 0 bits.

Over the last couple of years it has been openly vetted and tweaked and the
definition has been made significantly more robust. Alternatives were
considered, but nothing better has been implemented. This is a useful
process but one which has now run its course and is ready for a successful
conclusion: publishing of a stable open standard to facilitate wide
interop. That outcome does not preclude also working on something more
speculative.

The prototype protocol now carries over 1/3 of my HTTP transactions in
production - many others are waiting for a stable and open standard to work
with before joining the party. The only question left is really whether the
IETF is institutionally capable of providing that standard or another venue
will have to step up and fill that gap in this case. I certainly hope it
can - we'll find out soon.

-Patrick

---

From: Patrick McManus <[email-masked]>
Date: Tue, 2 Sep 2014 11:28:00 -0400

Hi Roy,

On Sun, Aug 31, 2014 at 7:06 PM, Roy T. Fielding <[email-masked]> wrote:

>
>
> The reason I did not go into any more detail than that is because the
> priority information is only a suggestion, like in h2.  Defining how
> the recipient is going to reprioritize its stack based on existing
> streams is an implementation detail behind the HTTP interface: wishful
> thinking and untestable (because it is only a suggestion).  Editorially,
> section 5.3 belongs in an appendix, or as a separate spec that has time
> to ruminate on the balance between weighted streams and various
> response characteristics.
>

Editorially and organizationally I think you're right but am happy to defer
to the editor. As a matter of content, I think the fairly complicated
scheme is well justified in terms of identified problems spdy has shown
with using just a few bits of weight (aggregation of lots of clients, dash
like dependencies, etc..)

>
> > 6.2.  HEADERS
> >
> Priority is a suggestion.

imo it is only really a suggestion in the sense that it is untestable.
Priority is a core piece of h2, because h2 introduces mux and a
non-prioritized mux is worse than h1.

>   Sending it only within a separate frame
> type incurs an additional overhead of 64bits, only when it is sent,
> assuming we also allow PRIORITY frames to initiate a stream.
>

roberto suggested basically that not too long ago - tracked as
https://github.com/http2/http2-spec/issues/347 ..

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 02 Sep 2014 15:37:52 +0000

--------
In message <CAOdDvNqqrqU-G3f7TuLDtY5+Pa1X=[email-masked]>
, Patrick McManus writes:

>Some folks can't be bothered to generate running code or sometimes even comment

It was obvious from the start that this was a "goldplate SPDY"
exercise and I know that I was not the only person who prioritized
my time accordingly.

If the process had started out with an open call to ask what the
goals for HTTP/2 should be in the first place it would be different,
but as it is, there suddenly existed a goal which was SPDY-shaped
and an aggressive schedule which made sure that nobody else could
challenge the obvious winner.

Feel free to agree with me, but I've had this percetion from day
one, and I have seen nothing that would make me change my mind.

---

From: James M Snell <[email-masked]>
Date: Tue, 2 Sep 2014 08:51:02 -0700

On Tue, Sep 2, 2014 at 8:25 AM, Patrick McManus <[email-masked]> wrote:
>[snip]
> conclusion: publishing of a stable open standard to facilitate wide interop.
> That outcome does not preclude also working on something more speculative.
>

Small correction: s/open standard/RFC

It's an important distinction. An RFC is still just a *Proposed*
Standard. Normally, this distinction wouldn't matter all that much but
in the case of a protocol as broadly used as HTTP 1.1, I believe it's
well worth mentioning.

> will have to step up and fill that gap in this case. I certainly hope it can
> - we'll find out soon.
>

So, in other words, "If you don't play how we want to play, we'll just
take our ball somewhere else!" That hardly seems helpful or "open".

Yes, this WG has worked for two and a half years on this draft, and in
some scopes, that can be considered quite a long time. I'm not certain
that any more work in this forum would really make the protocol better
either. However, here we have some very legitimate, non-editorial
feedback being submitted to WG by one of the broadly recognized
industry experts on HTTP questioning decisions that have come up
repeatedly over the past two and a half years. Personally, I'd
consider that to be a very strong signal that maybe the WG really
ought to take a step back for a second, listen and reevaluate some of
those decisions. The only question left is not whether the IETF is the
right venue, but whether the members of this WG are willing and
capable of putting ego aside long enough to consider that feedback
appropriately -- that is, after all, why we have a Last Call process
in the first place.

As far as Roy's feedback is concerned: My personal preference would be
for Priority to be removed from the main document entirely, and have
it pushed into a separate document as an optional extension. I think
Roy's suggestion of making it an HTTP header makes sense and provides
a great deal more flexibility in options down the road.

- James

---

From: Cory Benfield <[email-masked]>
Date: Tue, 2 Sep 2014 17:06:55 +0100

> term or the people (I too am guilty at times.)

I find this argument to be utterly bizarre. If we've spent 2.5 years
in committee and we turn out a protocol that's fatally flawed I'd
argue that that is a failure far worse than spending 5 years in
committee and getting something actually useful. If the opinion of the
WG is that h2-14 is fatally flawed then charging ahead for the sake of
finishing is madness. I am not the first person to make this
observation about a WG: Peter Hintjens wrote an extensive article on
this topic about the work done on AMQP in 2008[1].

Note that I don't believe h2-14 to be fatally flawed. *Flawed*,
certainly, but not fatally so. It solves a specific set of problems
and appears to solve them fairly well. That is a success by some
measure of success. I believe if we shipped h2-14 now it would be
adopted and used reasonably widely. It is not clear to me what the
definition of 'good enough' is, but it's clear to me that much of the
WG believes that bar is met. That portion of the WG is responsible for
assuaging Roy's concerns. Step up, y'all.

[1] http://www.imatix.com/articles:whats-wrong-with-amqp/

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 2 Sep 2014 18:12:50 +0200

Hi Pat,

> term or the people (I too am guilty at times.)

To be fair, there hava always be about two camps here :
  - those who could progress fast because they already had a running SPDY
    implementation that they could start from (and fortunately we had them) ;

  - those who had never implemented SPDY and for whom even participating in
    a way to bring some *value* is extremely hard because it's impossible
    to provide data nor to experiment, so they can only run aside the fast
    moving train trying to catch it.

I've experienced this period with some great frustration due to that but at
least I'm happy that the WG has made significant progress. However I would
have loved to have more chances of being active. I know can't blame anyone
for this except myself. But I also know that by the time it takes to completely
change an architecture to adapt to a new proposal (est. 1yr), it will have
completely changed to a point that such low-level architecture will be
obsolete again :-/

Thus I mean it's natural that when starting with an existing design, all
participants do not catch up at the same rate depending on the effort that's
required for them to have something to play with.

Just my two cents. Now I'm back to my work trying to rearchitect haproxy
to support multiplexed streams :-/

Willy

---

From: Patrick McManus <[email-masked]>
Date: Tue, 2 Sep 2014 12:43:48 -0400

> take our ball somewhere else!" That hardly seems helpful or "open".

James, I do not appreciate you saying 'in other words' and using quotation
marks in attempt to paraphrase me. I said what I said. You are free to say
what you like. In any event, I strongly disagree with your characterization.

The open h2 process has resulted in innumerable changes from spdy. Many of
them are good - the total sum of them certainly did not require 2.5 years
to work out, but the process of being open may have.  However, the largest
change is probably the mistake to remove the mandatory TLS binding. But I'm
still here even though I deeply disagree with that. Indeed there have been
some big disagreements over the last couple of years and I don't recall
anyone walking away though I'm sure we all have certain things that would
make the end result not worth shipping.

As we move through the process the bar for making a change is raised -
we've talked about that a number of times.

To clarify my statements, I indicated that the world is moving beyond
HTTP/1 - the data shows that. We've also got an existence proof that the
folks involved in that transition have sought to participate in an IETF
forum to produce an open standard around that transition. The consensus of
the group was to base h2 on spdy making changes as necessary. That's where
we've been for 2.5 years. As I said, what's left to be seen is whether or
not an open standard actually comes out the other end of the process. We've
made it to LC - now we need the discipline to separate substantive from
nice-to-have.

The WG or IAB/IESG could of course choose to standardize something
different than h2-14, or even decide that the basic prioritized muxx'ed
properties of h2 on tcp aren't something that should be standardized no
matter how widely used. That judgment is an important part of their roles
as a standard is at least partially an endorsement. I don't mean to
pre-suppose its outcome. If no open standard is produced the action will
likely move elsewhere because the need will not have dissipated. In the big
picture that's OK, in the immediate picture its an insane waste of 2.5
years of energy. And no, I don't expect that will happen - we just need to
be focused down the home stretch.

all feedback is important. And here I speak generically - not about Roy's
emails (where I wish to comment on them specifically, I will comment on
them specifically as I have elsewhere in this thread :)) - Last Call isn't
the best time to re-open issues that have been tracked, discussed, and
resolved previously.

---

From: James M Snell <[email-masked]>
Date: Tue, 2 Sep 2014 10:27:33 -0700

> what you like. In any event, I strongly disagree with your characterization.

Regardless of whether you agree with the characterization or not,
that's precisely how your first note cames across.

> emails (where I wish to comment on them specifically, I will comment on them
> specifically as I have elsewhere in this thread :)) - Last Call isn't the
> best time to re-open issues that have been tracked, discussed, and resolved
> previously.

It's my turn to disagree here. The point of last call is to broaden
the review to a much larger community. It's entirely reasonable and
expected to receive feedback on issues that had been tracked,
discussed and resolved previously by the WG. In fact, I think that's
pretty much a given.

- James

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 02 Sep 2014 17:37:56 +0000

--------
In message <[email-masked]>
, Patrick McManus writes:

>HTTP/1 - the data shows that.

I think you suffer badly from selection bias when you make this
statement.

*Your* world may be moving in that direction, but there are huge
swatches of HTTP traffic that doesn't seem to follow you and your
little red wagon.

IMO this effect has been present throughout, where representatives
from a small number of large sites seem to confuse their site and
other sites of the kind with "the world".

The whole "Mandatory TLS" is probably where we saw this most clearly:

The advocates of "Mandatory TLS" all seem to be from sites which
require user login and which have privacy data and concerns.

But those sites are not even close to carrying 50% of the HTTP
traffic in the net today:  The majority of traffic is public
without privacy concerns that is not already revealed by the
existence of the TCP connection in the first place.

I share the anti-NSA sentiment as much as the next guy, but there
is never going to be a sane case to be made why CNN or BBC's
frontpage has to suffer the huge overhead of TLS.

Nor is there any sane architectural argument to be made why
the emergency-services web-pages of national governments
should risk DoS'ing themselves with TLS during a catastrophy.

And there is no way the porn industry is ever going to fork out
money for TLS hardware for the 30+% of the total HTTP traffic they
serve.

So yes, you may *think* you have felt the world move, but I think
it is just the faulty suspension of your office-chair.

Poul-Henning

---

From: Roberto Peon <[email-masked]>
Date: Tue, 2 Sep 2014 10:47:05 -0700

This conversation is quickly becoming unproductive and diverging from the
technical issues.
Please refrain from venomous speech like 'follow you and your little red
wagon'.

If you have a different opinion, state it.
If you have a constructive criticism, say it.
If you intend to stir the pot for other reasons, please shut the hell up.

-=R

On Tue, Sep 2, 2014 at 10:37 AM, Poul-Henning Kamp <[email-masked]>
wrote:

> --------
> In message <
> [email-masked]>

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Sep 2014 20:44:47 +0200

On 2014-09-02 19:27, James M Snell wrote:
> ...
> ...

Reminder: this is "just" the WGLC. There will be another LC, the 
IETF-wide one. Do not assume that there won't be any new feedback at 
that stage.

Best regards, Julian

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Sep 2014 09:08:36 +1000

> for SETTINGS space and error codes space)[8].

Ilari,

while I do not agree with absolutely everything in your breakdown, I
definitely agree with the basic approach you are setting.

The fundamental issue I see here is that there is no separation between
HTTP semantics and the framing layer.  I believe this is the meta-issue
that we should debate and decide if we wish to solve it or not.     Until
we have decided that fundamental question, I think we are bound to repeat
many of the technical debates we have had without a resolution that
satisfies a significant majority.

This working group started with a choice between two starting points: speed
and mobility (HTTP semantics over websocket framing) and SPDY (HTTP
semantics over a new multiplexed framing layer).    Together with the
charter to consult websocket, it looked like it was pretty clear that the
original intend was to support multiple semantics.

But somewhere along the way, this clear separation was dropped and I
believe most of the current disagreements stem from that decision.

Rather than continue to debate "this is a poor design for a framing layer"
vs "it works perfectly well for HTTP",  I think we need to step back and
have a meta discussion.     Is it an issue or not that the framing layer is
HTTP specific?

regards

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 2 Sep 2014 16:43:08 -0700

I would really appreciate it if folks would stop hijacking my threads
in order to make their own orthogonal proposals, grandiose or small, or
to make general commentary about process, or to begrudge the lack of
adoption of some past comment.  That isn't right.

If you have comments on my suggestions, good or bad, then please make
them on the same thread.  If you want to go beyond them, that's fine
only if it stays within the scope of the thread.  Otherwise, start a new
thread so that it can be clear what proposal is being discussed.

....Roy

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Sep 2014 10:32:13 +1000

> I would really appreciate it if folks would stop hijacking my threads

Sorry my bad: anticipating failure again.

On the substance of this thread, I agree that an optional priority should
not be part of headers frame.  We've not yet implemented priorities but are
interested in experimenting with their use.   Happy for the detail to be
moved to appendix or other document.

regards

---

From: James M Snell <[email-masked]>
Date: Tue, 2 Sep 2014 17:45:06 -0700

+1 to the notion of moving priority out of the http2 doc and into a
separate document.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Sep 2014 09:42:32 +0300

Roy,

It seems like the possibilities you map out are:

1. Make Priorities a request header field (removing it from HEADERS)

This isn’t a complete solution, because we’ve also identified the need to reprioritise existing streams. So, we’d still need a separate frame type; however, this would allow priority to be moved out of HEADERS.

Intermediaries would need to parse the header (but they already need to parse the pseudo-headers, so that’s not necessarily a deal breaker) and set the header for upstream traffic (which arguably is more expensive than keeping it in the frame header).

2. Require priority to be sent in a PRIORITY frame (removing it from HEADERS)

As you point out, this would incur the overhead of an additional frame; AIUI browsers want to set priority for every request, so this would effectively double the frame overhead for all of their requests.

Yet again, I don’t see concrete interoperability problems here — just a stated preference for a change. As we’ve said before, the bar for such changes has been raising considerably as we’ve worked through the process, and at this late stage, we really need to see broad support for a change to make it in, absent real security or interoperability problems (before you ask, this has been confirmed with our AD).

So, what do people think?

On 1 Sep 2014, at 2:06 am, Roy T. Fielding <[email-masked]> wrote:

> This comment is in reference to sections 5.3, 6.2, and 6.3 of
> 
> http://tools.ietf.org/id/draft-ietf-httpbis-http2-14.txt
> 
>> 5.3.  Stream priority
>> 
>>   A client can assign a priority for a new stream by including
>>   prioritization information in the HEADERS frame (Section 6.2) that
>>   opens the stream.  For an existing stream, the PRIORITY frame
>>   (Section 6.3) can be used to change the priority.
>> 
>>   The purpose of prioritization is to allow an endpoint to express how
>>   it would prefer its peer allocate resources when managing concurrent
>>   streams.  Most importantly, priority can be used to select streams
>>   for transmitting frames when there is limited capacity for sending.
>> 
>>   Streams can be prioritized by marking them as dependent on the
>>   completion of other streams (Section 5.3.1).  Each dependency is
>>   assigned a relative weight, a number that is used to determine the
>>   relative proportion of available resources that are assigned to
>>   streams dependent on the same stream.
>> 
>>   Explicitly setting the priority for a stream is input to a
>>   prioritization process.  It does not guarantee any particular
>>   processing or transmission order for the stream relative to any other
>>   stream.  An endpoint cannot force a peer to process concurrent
>>   streams in a particular order using priority.  Expressing priority is
>>   therefore only ever a suggestion.
>> 
>>   Prioritization information can be specified explicitly for streams as
>>   they are created using the HEADERS frame, or changed using the
>>   PRIORITY frame.  Providing prioritization information is optional, so
>>   default values are used if no explicit indicator is provided
>>   (Section 5.3.5).
> 
> (editorial) The first sentence of last paragraph is a duplicate of the
> first paragraph of 5.3.
> 
> [... four pages of details on reprioritization based on weighted trees ...]
> 
> Does anyone implement this?  I have to wonder, because it appears smack
> in the middle of the most important part of the spec and seems to have
> no relevance to anything else in the protocol.  The description is very
> interesting from an academic perspective (and I like that), but I don't
> see how any of this is supposed to work in a live environment where
> streams exist for a single request/response. Why bother?
> 
> As I described in Paris, waka has a couple bits in every request to
> express similar priority as one of four relative choices:
>   (speculative, low, HiLo, high)
> where speculative is essentially zero priority for things like prefetch
> (don't answer until you have done all the other work) and HiLo means
> the client needs the rendering metadata right away but the remainder,
> as determined by the sender, can be lower priority.
> 
> The priority in waka is just expressing how important the response is
> to the requester relative to the other requests they have made.
> 
>>   The HEADERS frame (type=0x1) carries name-value pairs.  It is used to
>>   open a stream (Section 5.1).  HEADERS frames can be sent on a stream
>>   in the "open" or "half closed (remote)" states.
>> 
>>     0                   1                   2                   3
>>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>    |Pad Length? (8)|
>>    +-+-------------+-----------------------------------------------+
>>    |E|                 Stream Dependency? (31)                     |
>>    +-+-------------+-----------------------------------------------+
>>    |  Weight? (8)  |
>>    +-+-------------+-----------------------------------------------+
>>    |                   Header Block Fragment (*)                 ...
>>    +---------------------------------------------------------------+
>>    |                           Padding (*)                       ...
>>    +---------------------------------------------------------------+
>> 
>>                           HEADERS Frame Payload
> 
> YIKES!  I already mentioned padding in a prior message, so let's just
> assume that has been deleted.
> 
> Here we have 40 optional bits that might or might not exist as a
> preamble to the really important data that everyone needs to see first
> in order to do any sort of request routing.  Sorry, that's nuts.
> 
> Priority is a suggestion.  Sending it only within a separate frame
> I think that is a better trade-off than the current spec.
> 
> Sending priority as 2 bits in the frame itself is a better idea, IMO,
> but I won't push the WG in that direction.
> 
> Sending it as optional bits in front of the headers means that the
> most important bits of the protocol are going to be shifting around
> in the buffers depending on the value of the PRIORITY bit.  Not good.
> 
> If we really want to set priority within a header frame, why don't
> we just make it a header field?  A server won't be changing any
> priorities until it knows what has been requested, regardless.
> 
> 
> Cheers,
> 
> Roy T. Fielding                     <http://roy.gbiv.com/>
> Senior Principal Scientist, Adobe   <http://www.adobe.com/>
> 
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Sep 2014 17:03:06 +1000

On 3 September 2014 16:42, Mark Nottingham <[email-masked]> wrote:

>
> As you point out, this would incur the overhead of an additional frame;
> AIUI browsers want to set priority for every request, so this would
> effectively double the frame overhead for all of their requests.
>

So surely priority should be sent as a HTTP header like all other request
meta data?
For pushed responses, priority could be adjusted with a stream setting.

> So, what do people think?

That there are a lot of issues that many would try to get over a lowered
bar.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 03 Sep 2014 07:10:52 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>2. Require priority to be sent in a PRIORITY frame (removing it from =
>HEADERS)

As much as I would hate if we did it, there is the option of two
frame formats:

Short frames have no payload, and the length field can be used for
24 bits of information.

Long frames have a payload and the length field tells how long it is.

In addition to PRIORITY frames, that could shave 4 bytes of all
WINDOW_UPDATES

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 03 Sep 2014 23:44:59 +1200

On 3/09/2014 6:42 p.m., Mark Nottingham wrote:
> This isn’t a complete solution, because we’ve also identified the
> need to reprioritise existing streams. So, we’d still need a
> separate frame type; however, this would allow priority to be moved
> out of HEADERS.
> 
> Intermediaries would need to parse the header (but they already
> need to parse the pseudo-headers, so that’s not necessarily a deal
> breaker) and set the header for upstream traffic (which arguably is
> more expensive than keeping it in the frame header).
> 
> As you point out, this would incur the overhead of an additional
> frame; AIUI browsers want to set priority for every request, so
> this would effectively double the frame overhead for all of their
> requests.
> 
> Yet again, I don’t see concrete interoperability problems here —
> just a stated preference for a change. As we’ve said before, the
> bar for such changes has been raising considerably as we’ve worked
> through the process, and at this late stage, we really need to see
> broad support for a change to make it in, absent real security or
> interoperability problems (before you ask, this has been confirmed
> with our AD).
> 

That I've witnessed a large number of little bits and pieces of
annoyance complained about repeatedly which individually do not make
som earbitrary "bar". But the nature of annoyances like these is that
many little ones accumulate into a big problem.

Perhapse WGLC is a good time for a re-count by chair and editors to
see if the total of the small improvements add up to enough for a mass
fix?

Amos

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 3 Sep 2014 15:15:20 -0700

Hi Patrick,

Sorry, I almost missed this one in the noise ...

On Sep 2, 2014, at 8:28 AM, Patrick McManus wrote:
> Editorially and organizationally I think you're right but am happy to defer to the editor. As a matter of content, I think the fairly complicated scheme is well justified in terms of identified problems spdy has shown with using just a few bits of weight (aggregation of lots of clients, dash like dependencies, etc..)

Does that mean it has been implemented?  I mean, does the weighted tree
with stream dependencies result in effective prioritization by a server?

I am curious about that because I see most UA traffic as parallel with
aborts, rather than dependent streams, and expect a user agent to prioritize
by fairly well-understood latency buckets (critical path, js, renderable
content, not-rendered-yet content, background prefetch, long pull, etc.).
IOW, I don't see how a client can guess which streams are "more important"
other than by the perceived latency buckets.

> imo it is only really a suggestion in the sense that it is untestable. Priority is a core piece of h2, because h2 introduces mux and a non-prioritized mux is worse than h1.

Well, worse than h1 + multiple connections.  Yes, that's why I thought
priority would be important as well.  What I found disconcerting
is that the spec takes a deep dive into explaining how priorities work,
before it even defines the frames, and yet doesn't actually use any of
that stuff in examples or later prose.

....Roy

---

From: Patrick McManus <[email-masked]>
Date: Thu, 4 Sep 2014 12:57:41 -0400

Hi Roy,

So the priority dependencies is really a feature googlers championed, I'm
surprised that we haven't seen anyone from that team contributing to this.
I do, however, support it so I'll carry on the discussion.

The most important bit of consensus was that experience has shown us the
few bits of spdy priority can't be aggregated well. The easiest
illustration is a proxy...

if 2 inbound sessions have 3 streams each of priorities {0,1,2} and {5, 6,
7} all headed to the same origin what is a reasonable implementation for
the priorities on the 1 outbound session? The incoming priorities only have
meaning wrt other streams in the same session, they have no global meaning.
Each client is incented to pick priorities at the most important end of the
spectrum - indeed perhaps to only use the highest priority which is back to
having no priority at all.

So that was the core issue that motivated a more complicated scheme, and I
think there was strong support around that. And of course things other than
proxies aggregate.. OS's.. even browsers between tabs and plugins are
basically doing the same function sometimes with more context than others.

After that you get into more speculative use cases for sure. I've got one
whiteboarded regarding dependencies for images based on what the layout
engine says might be visible at any moment in time, but its stalled mostly
on non-network internal browser stuff of figuring out what might be visible
at any moment in time :) For the most part I agree we are basically
speculating on what might work in that space right now, but the previously
mentioned aggregation bits were a known problem that the text sought to
address.

hth

On Wed, Sep 3, 2014 at 6:15 PM, Roy T. Fielding <[email-masked]> wrote:

> Editorially and organizationally I think you're right but am happy to
> defer to the editor. As a matter of content, I think the fairly complicated

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 4 Sep 2014 10:49:39 -0700

On Sep 4, 2014, at 9:57 AM, Patrick McManus wrote:

> Hi Roy,
> 
> So the priority dependencies is really a feature googlers championed, I'm surprised that we haven't seen anyone from that team contributing to this. I do, however, support it so I'll carry on the discussion.
> 
> The most important bit of consensus was that experience has shown us the few bits of spdy priority can't be aggregated well. The easiest illustration is a proxy...
> 
> if 2 inbound sessions have 3 streams each of priorities {0,1,2} and {5, 6, 7} all headed to the same origin what is a reasonable implementation for the priorities on the 1 outbound session? The incoming priorities only have meaning wrt other streams in the same session, they have no global meaning. Each client is incented to pick priorities at the most important end of the spectrum - indeed perhaps to only use the highest priority which is back to having no priority at all.

Well, I can't say much about SPDY priorities because they are just numbers.
The two bits that I used in waka benefit from having fewer choices since
that is easy to understand and partition between shared connections.
What happens is that the recipient splits traffic equally among multiple
streams with the same priority and the proxy pumps up the priority of a
stream if one of the two (or more) clients is being starved (which should
only occur if that client has only zero-priority streams remaining).

There is still nothing to prevent any one client from getting greedy and
asking for high priority all of the time, but a server is not required
to obey the priorities.  A server might be setting its priorities based on
source behavior (e.g., AOL proxies were a common target for unusual
server settings) or on past browser statistical profiles (the same ones an
h2 server might use to determine when push_promise is useful).

> So that was the core issue that motivated a more complicated scheme, and I think there was strong support around that. And of course things other than proxies aggregate.. OS's.. even browsers between tabs and plugins are basically doing the same function sometimes with more context than others.
> 
> After that you get into more speculative use cases for sure. I've got one whiteboarded regarding dependencies for images based on what the layout engine says might be visible at any moment in time, but its stalled mostly on non-network internal browser stuff of figuring out what might be visible at any moment in time :) For the most part I agree we are basically speculating on what might work in that space right now, but the previously mentioned aggregation bits were a known problem that the text sought to address.

Okay, thanks for the info.  I'd be happier if there were a real example
of stream dependencies working in practice, since that is 32 bits.
Are stream dependencies intended to reflect tabs?

....Roy

---

From: Patrick McManus <[email-masked]>
Date: Thu, 4 Sep 2014 16:21:40 -0400

> Are stream dependencies intended to reflect tabs?

That's certainly one use case offered up.. another common one is sequential
media chunks (ala dash)