Subject: http/2 prioritization/fairness bug with proxies

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 4 Feb 2013 14:33:23 +0900

Mike told me I didn't explain this properly at the interim meeting,
which was totally true, since I was just trying to do a brief survey
of browser considerations. In retrospect, I'll prepare fuller
presentations next time to explain things more clearly.

Anyway, the existing prioritization bug is as follows:
* Multiple users speaking HTTP/2 to a proxy, where they indicate
stream priorities within their respective HTTP/2 sessions
* The proxy speaks HTTP/2 to a server, demuxing the client sessions
and re-muxing some of the streams into a shared HTTP/2 session to a
server.

The natural thing to do in HTTP/2 as currently drafted is to have the
proxy simply respect the clients' priorities when forwarding to the
server. That obviously means that specific clients can request
long-lived high priority streams, or repeatedly request high priority
streams. This may or may not starve other streams, depending on how
the backend server handles the priorities.

There are a number of different ways to handle this in HTTP/2 as
currently drafted:
* Long-lived high priority streams can be slowly deprioritized by the
backend server.
* The proxy can modify the priorities as it sees them. It could
neutralize them all (set them all to equivalent values) or if a client
requests too many high priority streams, it could start lowering the
priority levels of new streams from that client. The backend server
obviously can't do this because it doesn't (at least, shouldn't!) know
the clients behind the proxy.
* The proxy can use separate HTTP/2 sessions for each client.

I consider all those options as suboptimal, and thus consider this
issue to be a protocol bug. Our SPDY/4 prioritization proposal
addresses this by using stream groups with advisory (all this is
advisory after all) per group weights (for weighted scheduling). I'd
like to hear what people think of this issue and how we should address
it in HTTP/2.

Cheers.

---

From: Yoav Nir <[email-masked]>
Date: Mon, 4 Feb 2013 06:26:33 +0000

Hi

I think the third option (separate streams for each client) is the most fair. Whatever mechanism is used by the server for fairness among clients will be at work here. What is the downside here, wasting TCP sockets?

What if we added a header (or a field to a header) called proxy-disambiguation, where the proxy assigns a short (16-bit? 32-bit?) identifier for each real client. That way the issue of prioritization/fairness gets pushed to the server where it belongs.

Where is the SPDY/4 prioritization proposal described?

Yoav

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Feb 2013 17:51:13 +1100

Thanks, William. I think most of this came out in discussion.

Personally, as someone who works with proxies, I like grouping (as I said in the meeting).

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 04 Feb 2013 06:57:02 +0000

Content-Type: text/plain; charset=ISO-8859-1
--------
In message <CAA4WUYjiBZpShKKFfHQnixc94aOLrck0oR4ykARB=[email-masked]>

>Anyway, the existing prioritization bug is as follows:

[...]

>The backend server

This flies counter to vast experience:  Servers do almost everything
they can to identify the actual client (X-F-F, cookies, fingerprinting
etc), so I think this premise needs to be rethought.

>advisory after all) [...]

So what does the groups buy you, for all the complexity they add ?

As far as I can tell:  Nothing.

I think the priority should simply be documented as advisory and
mention that intermediaries SHOULD respect them, subject to
local administrative policy, and leave it at that.

The current proposal is complex enough as it is, adding complexity
to not solve problems that cannot be solved technically, is not
an improvement.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Ilya Grigorik <[email-masked]>
Date: Sun, 3 Feb 2013 23:56:23 -0800

> Where is the SPDY/4 prioritization proposal described?

Original thread from spdy-dev:
https://groups.google.com/d/msg/spdy-dev/-d9Auoun4HU/Pu73zcGv8eMJ

Also, original doc:
https://docs.google.com/document/d/1pNj2op5Y4r1AdnsG8bapS79b11iWDCStjCNHo3AWD0g/edit

One heads up though.. I believe the doc is out of date, and needs to be
updated. (in progress, I believe...)

ig

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 04 Feb 2013 21:35:26 +1300

> to not solve problems that cannot be solved technically, is not

+1. There are a lot of queuing, scheduling, and load balancing 
algorithms out there for the proxy to choose from and likely to be more 
or better ones found going forward.
The proxy is already in a position to group (or not), or to simply delay 
I/O as necessary to make things fair for all clients according to their 
priority hints. A SHOULD is enough.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Feb 2013 21:01:50 +1100

On 04/02/2013, at 7:35 PM, Amos Jeffries <[email-masked]> wrote:

> +1. There are a lot of queuing, scheduling, and load balancing algorithms out there for the proxy to choose from and likely to be more or better ones found going forward.
> The proxy is already in a position to group (or not), or to simply delay I/O as necessary to make things fair for all clients according to their priority hints. A SHOULD is enough.

So, let's say I'm a proxy, and I currently have two clients, A and B.

Client A sends a stream of requests with the following priorities:

1 1 1 3 3 3 2 2 

and client B sends:

3 3 2 2 4 4

Let's say (for now) that they all go on the same forward connection.

With the current mechanism, you have to rewrite the priorities to balance their connections. So, you might change B's request stream priorities to:

2 2 1 1 3 3

But what should I do when, a few milliseconds later, B emits a request with priority "1"? I don't have a "0" priority. 

I suppose I could de-prioritise client A's requests, but that would just move the problem to the other end of the scale (least priority).

I imagine that a clever implementer would find a way to re-balance the priority space so that these effects were mitigated. However, not all implementers are clever, and even the clever implementers will do it in different ways, leading to variations in behaviour.

To me, grouping seems like a very easy way around this; you can maintain the relative priority of requests from a single context without rewriting them, and level out the weightings of groups if you're a proxy. 

BTW, I think there's a similar motivation in the original use case that motivated Will; he needs to be able to balance the priorities among a set of browser tabs, when it's possible for new requests to be generated by those tabs at any time.

Cheers,

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 05 Feb 2013 02:02:45 +1300

> Client A sends a stream of requests with the following priorities:

Which goes to show why specifying an algorithm for handling opaque 
client-selected prioritization is the wrong way to go about this.

You have presented a good argument for specifying a set of standardized 
priority labels with criterion for setting each label.
eg.
  Priority 1 - user actioned fetch - requires fast answer
  Priority 2 - background/automated fetch in user-visible window - 
requires fast answer but treat as bulk traffic.
  Priority 3 - automated software fetch - treat as low-speed traffic.
  Priority 4 - idle software probe - may drop if necessary.

... then letting the algorithm designers and implementers free to write 
their prioritization algorithms around those definitions.

Amos

---

From: Albert Lunde <[email-masked]>
Date: Mon, 04 Feb 2013 09:56:01 -0600

On 2/4/2013 7:02 AM, Amos Jeffries wrote:
  Which goes to show why specifying an algorithm for handling opaque
> priority labels with criterion for setting each label.

One possible gotcha there is that not everything is a browser. A server 
application issuing, say, web service requests may have different 
pasterns of usage than a browser client used by an typical human user.

On another hand, the example of multiple TCP connections in HTTP/1.1 and 
in particular, "download accelerators", illustrates that some client 
software will be developed to "game the system" any way it can; it's 
going to be hard to be fair.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 5 Feb 2013 01:04:28 +0900

On Feb 3, 2013 10:26 PM, "Yoav Nir" <[email-masked]> wrote:
>
> Hi
>
> I think the third option (separate streams for each client) is the most
fair. Whatever mechanism is used by the server for fairness among clients
will be at work here. What is the downside here, wasting TCP sockets?

Using separate connections is generally suboptimal for all the reasons we
are adding multiplexing.

>
> What if we added a header (or a field to a header) called
proxy-disambiguation, where the proxy assigns a short (16-bit? 32-bit?)
identifier for each real client. That way the issue of
prioritization/fairness gets pushed to the server where it belongs.

That's an option if we agree we should change the protocol to support
stream grouping. I'm focusing now on that decision, not on the choice of
the actual mechanism.

> Where is the SPDY/4 prioritization proposal described?

I didn't link to it earlier because I wanted to focus on the stream
grouping concept and not the other aspects of the proposal.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 5 Feb 2013 01:42:14 +0900

On Mon, Feb 4, 2013 at 5:35 PM, Amos Jeffries <[email-masked]> wrote:
>> In message
>> <CAA4WUYjiBZpShKKFfHQnixc94aOLrck0oR4ykARB=[email-masked]>

Sorry, I wasn't clear. I meant within the protocol itself. You're
absolutely correct that there are methods of identifying particular
clients. That said, as Mark notes, one of the original use cases
motivating this discussion is the tabbed browser case. I like to think
of a browser network stack as a proxy, servicing network requests for
the various tabs. While other methods do exist to handle identifying
particular tabs, they aren't particularly good ones.

But let's return to the premise. Do you believe it's better to rely on
these client fingerprinting methods to implement prioritization,
rather than providing more explicit signals in the protocol itself?

>> As far as I can tell:  Nothing.

I'm sorry I didn't address this in the first email. I confess I
thought it was obvious. Grouping lets you do relative prioritization
within a group, as opposed to across the entire session.

> +1. There are a lot of queuing, scheduling, and load balancing algorithms
> out there for the proxy to choose from and likely to be more or better ones
> found going forward.
> The proxy is already in a position to group (or not), or to simply delay I/O
> as necessary to make things fair for all clients according to their priority
> hints. A SHOULD is enough.

AFAICT, delaying I/O is a suboptimal solution. Since client-side
advisory priorities informs the server of the response ordering, a
proxy which passes through the same priorities will allow higher
priority stream responses to starve lower priority ones. In your
proposed solution of delaying I/O, the proxy would need to withhold
WINDOW_UPDATE frames for streams that it wants to deprioritize, in
order to prevent the server from sending the response data for the
higher priority streams. In effect, this uses the stream receive
window mechanism to implement prioritization. I think that having the
receiver implement prioritization via receive windowing is worse than
having the client inform the server of advisory priorities and letting
the sender (the server) order as appropriate.

---

From: Nico Williams <[email-masked]>
Date: Mon, 4 Feb 2013 10:46:52 -0600

On Feb 4, 2013 8:05 AM, "Amos Jeffries" <squid3
<[email-masked]>@<[email-masked]>

+1, plus priorities for real-time requests?

---

From: Nico Williams <[email-masked]>
Date: Mon, 4 Feb 2013 10:49:09 -0600

---

From: James M Snell <[email-masked]>
Date: Mon, 4 Feb 2013 09:09:19 -0800

Prioritization is, admittedly, no where near my strongest area so if this
is a silly notion please do not hesitate to point it out... Just throwing
this in off the top of my head, but, could we not allow for priority quotas
per session? Allow an intermediary to open a limited number of concurrent
connections with the server, each with a specific set of priority quotas.
If an intermediary is remuxing requests, it would pass those through
without modifying the priorities, but could choose which connection to send
it along on based on the available quota. Certain connections would be
weighted to give the highest quota to high priority streams, others
weighted for lower priority.
On Feb 4, 2013 8:50 AM, "Nico Williams" <[email-masked]> wrote:

>
> On Feb 4, 2013 8:05 AM, "Amos Jeffries" <squid3 <[email-masked]>@<[email-masked]>

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 4 Feb 2013 09:49:54 -0800

Note that these priority labels would be suboptimal for browser usage.
As I understand it, all requests within a page load would fall within
a single priority level, probably priority 1 in your case. In my
presentation, I demonstrated how browsers want to do resource request
prioritization within the HTTP protocol level. Different resources
have very different prioritization requirements within a page load,
and unless the protocol supports client advisory priorities, the
browser must make a link underutilization vs contention tradeoff.
Specifically, the browser will withhold making HTTP requests for low
priority resources under higher priority HTTP requests complete, so as
to avoid contention. This achieves some amount of prioritization at
the risk of underutilizing the link.

As I understand it, your proposal also requires trustworthy clients to
set the appropriate priority level, unless the proxy has some
mechanism for determining the appropriate priority level by inspecting
the stream, at which point the client requested priority is useless.
I'm not sure that trusting clients to set the appropriate priority
level for shared sessions is advisable. Opaque levels where only
relative priority matters within a "group" is resilient to this
problem.

On Mon, Feb 4, 2013 at 8:46 AM, Nico Williams <[email-masked]> wrote:
>
> On Feb 4, 2013 8:05 AM, "Amos Jeffries" <[email-masked]> wrote:
>>  Priority 2 - background/automated fetch in user-visible window - requires
>> fast answer but treat as bulk traffic.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 04 Feb 2013 18:29:38 +0000

Content-Type: text/plain; charset=ISO-8859-1
--------
In message <CAA4WUYiBJrLjM0-vurFOuJfUaabXtK=[email-masked]>

>rather than providing more explicit signals in the protocol itself?

I belive that the client is welcome to suggest what kind of priority
it would like to have, and that in the end, the other end is going
to do whatever it likes, which may be to honor the clients desires,
but most often isn't.

>within a group, as opposed to across the entire session.

I understood that, my question pertains to reality:  What do you
get in the _real_ world scenario ?

Likely a DoS amplification if the proxy honors the client's priority
desires...

I'm fine with the client communicating a desired priority, I'm not
fine with 50 pages of standards-verbiage about what the other end
should do about it.

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Mon, 4 Feb 2013 20:26:52 +0000

Poul-Henning,

> fine with 50 pages of standards-verbiage about what the other end

I believe what is being suggested is:

- The spec includes some more 'hooks' for communicating more than just a single set of priorities (as is the case with SPDY today) based on feedback from Will etc from the limitations they've found in SPDY's current prioritisation options.
- Honouring of priorities and how exactly different priorities are handled/scheduled/etc is down to the implementation of the sender.

So the idea is the protocol contains enough 'hooks' to sufficiently express the different priorities between & within groups that folks would like to express but isn't prescriptive about how anyone uses or implements different prioritisation, scheduling, etc schemes.

HTH
Ben

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 04 Feb 2013 20:34:39 +0000

Content-Type: text/plain; charset=ISO-8859-1
--------
In message <[email-masked]>, Ben Nive
n-Jenkins writes:

>So the idea is the protocol contains enough 'hooks' to sufficiently 
>express the different priorities between & within groups that folks 
>would like to express but isn't prescriptive about how anyone uses or 
>implements different prioritisation, scheduling, etc schemes.

That was clearly not how the original poster presented it:

	"I consider all those options as suboptimal, and thus
	consider this issue to be a protocol bug. Our SPDY/4
	prioritization proposal addresses this by [...]"

---

From: Patrick McManus <[email-masked]>
Date: Tue, 5 Feb 2013 08:46:04 +0900

> have very different prioritization requirements within a page load,

 I just wanted to echo that this is a very important point.. the
relationship between css/js and images is the primary example but there are
others.

 >>. Opaque levels where only

> relative priority matters within a "group" is resilient to this
yes.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 5 Feb 2013 10:47:19 +0900

I'm sorry if I am unclear in any way. Please continue to
challenge/question my comments/assertions so I can clarify my position
as appropriate.

Just to be clear here, I stand by that it's a protocol bug currently.
I agree with adding more hooks to convey advisory priority semantics.
That said, "advisory" is open to interpretation. I agree that the
sender should ultimately be in control of how it orders responses, and
indeed there are of course many situations where it's best for the
sender to ignore the advisory priority. Yet, if the advisory priority
semantics are generally not respected, then clients will not be able
to rely on them, and will be forced to implement prioritization at a
higher layer, which suffers from the link underutilization vs
contention tradeoff I highlighted earlier.

I appreciate the concern that we're adding complexity by introducing
new semantics. I am arguing that because the existing mechanisms for
addressing starvation are suboptimal, we should treat this as a
protocol bug and thus change the protocol in such a way as to fix this
problem. My suggestion for doing so was adding new priority "grouping"
semantics. I am hopeful that these new semantics will not introduce an
inordinate amount of specification, as the primary idea is that the
current SPDY priority levels would apply within a "group". I think we
can come up with a way to define a group that will be relatively easy
to spec.

SPDY/4 introduces other prioritization semantics beyond just grouping,
but I wanted to focus on this one first, as I believe this is a bug
that we *need* to fix. The other SPDY/4 priority changes are of a
performance optimization nature, and I believe they will need to be
justified by data. I have no plans to raise them up in this group
until we have said data.

---

From: Ashok Kumar <[email-masked]>
Date: Fri, 8 Feb 2013 11:20:43 +0530

> Priority 4 - idle software probe - may drop if necessary.

I still feel fixed priorities have better use, from a server point of view.
The description may need to be tweaked a bit from the user-agent's needs
perspective. A number will always have different meaning from different
clients and server is not best place to guess who is more important.

One tangential question, please spare me if this is not the right place.
How is priority supposed to work from a server's perspective. Let's say
Apache has a MaxClient of 1000. Will it stop accepting normal requests
after it has used up 80% quota and will wait for higher priority requests?
May be a trivial implementation detail but this has been bothering me for
sometime and I thought this might help in deciding how priorities should be
sent/interpreted.

-- 
.- ... .... --- -.-

---

From: Roberto Peon <[email-masked]>
Date: Thu, 7 Feb 2013 23:23:43 -0800

It should always keep the pipe full, but it should be using small enough
frames that it can respond with alacrity to a higher priority resource.
So, in generally, the algorithm should be something like:

WriteToAStream():
  for priority-bucket in active-priority-list:
    for stream in priority-bucket:
      if is_readable(stream):
        outbound.WriteData(stream, 2**14)
        return

other variations are possible, but that is the basic idea.

On Thu, Feb 7, 2013 at 9:50 PM, Ashok Kumar <[email-masked]> wrote:

> I still feel fixed priorities have better use, from a server point of
> view. The description may need to be tweaked a bit from the user-agent's
> needs perspective. A number will always have different meaning from
> different clients and server is not best place to guess who is more
> important.
>

---

From: Mike Belshe <[email-masked]>
Date: Sun, 10 Feb 2013 14:06:25 -0800

> Just to be clear here, I stand by that it's a protocol bug currently.

+0.5.  :-)

Mark - I think we could open a issue ticket with the current HTTP/2.0 draft
that this is a bug which will present itself for servers that implement
naively.  This isn't strictly a "bug", since the spec says the server can
do whatever it wants with priorities, but this is subtle enough (surfacing
primarily in http/2 -> http/2 proxy situations), that many server
implementors won't think of it unless we mention it in the spec.

The bare minimum would be to simply document it and tell servers not to
starve out streams.  However, this is probably a wimpy approach and I think
we can do better.

The SPDY/4 prioritization changes are far more reaching than just fixing
this bug.  While I like grouping as a feature, I don't believe it actually
fixes this bug:  a browser could open a high priority, infinite stream,
which is competing across a shared proxy backend with other streams; unless
the user manually switches tabs (or does something to force changed
groups), the starvation can still occur.

Mike

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 10 Feb 2013 14:40:33 -0800

First, I totally agree SPDY/4 prioritization changes are far more reaching.
Let's not talk about them yet. I share your complexity concern and agree we
need data before proceeding with many of those features, and I plan on
getting data.

As far as grouping, I think it's definitely a bug for the case where you
have a forward proxy with users sharing the same HTTP/2.0 connection to an
origin server. You can have many high priority short-lived streams which
can starve out others, unless we implement the vague notion of "don't
starve out streams", which is difficult when it might be a sustained rate
of high priority short-lived streams. It's easier in your latter case of
infinite, high-priority streams.

You're right that the high priority, infinite stream can starve streams
within the same group. I don't think this means that grouping is not
required, but that grouping is potentially insufficient. For intragroup
starvation, I think it's debatable about whether the server should be smart
and not allow streams to _completely_ starve other streams within a group,
or that clients should have a reprioritization facility. I think this is a
discussion worth having, but I'd personally classify it as separate from
whether or not the grouping feature is necessary.

On Sun, Feb 10, 2013 at 2:06 PM, Mike Belshe <[email-masked]> wrote:

> Mark - I think we could open a issue ticket with the current HTTP/2.0
> draft that this is a bug which will present itself for servers that
> implement naively.  This isn't strictly a "bug", since the spec says the
> server can do whatever it wants with priorities, but this is subtle enough
> (surfacing primarily in http/2 -> http/2 proxy situations), that many
> server implementors won't think of it unless we mention it in the spec.
>

---

From: Martin Thomson <[email-masked]>
Date: Mon, 11 Feb 2013 14:39:41 -0800

Keep in mind that it is always possible for the intermediary to apply
flow control to the infinite length stream so that, regardless of
priority, it doesn't consume more than its fair share.

On 10 February 2013 14:40, William Chan (陈智昌) <[email-masked]> wrote:
> origin server. You can have many high priority short-lived streams which can
> starve out others, unless we implement the vague notion of "don't starve out
> streams", which is difficult when it might be a sustained rate of high
> priority short-lived streams. It's easier in your latter case of infinite,
> high-priority streams.
>
>> (surfacing primarily in http/2 -> http/2 proxy situations), that many server
>> the user manually switches tabs (or does something to force changed groups),
>> the starvation can still occur.
>>
>> Mike
>>
>>
>>
>>
>>>

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 11 Feb 2013 14:46:31 -0800

Theoretically possible is one thing. But the moment we get into the game of
trying to carve up portions of BDP via per-stream flow control windows for
prioritization purposes in normal operation (as opposed to just trying to
make reasonable progress under excessive load), I think we're in trouble,
and likely to get into performance issues due to poor implementations. As
I've stated before, I hope most implementations (and believe we should add
recommendations for this behavior) only use flow control (if they use it at
all, which hopefully they don't because it's hard) for maintaining
reasonable buffer sizes.

On Mon, Feb 11, 2013 at 2:39 PM, Martin Thomson <[email-masked]>wrote:

> can
> >> On Mon, Feb 4, 2013 at 5:47 PM, William Chan (陈智昌) <
> [email-masked]>
> >> wrote:
> >>>
> server
> >>> > In message <[email-masked]
> >,
> >>> > Ben Nive
> >>> > n-Jenkins writes:
> >>> >

---

From: Martin Thomson <[email-masked]>
Date: Mon, 11 Feb 2013 14:53:00 -0800

I'm thinking a simple mechanism.  The proxy can feed requests back to
clients based on its own prioritization and use flow control windows
on the back-end connections to ensure that it doesn't have to buffer
infinitely.  Basically, the same sorts of logic that would be needed
by a proxy that serves multiple clients.

Yes, this is sub-optimal because it leads to either poor bandwidth
utilization, lots of buffering, or worse.

Sure, it's very easy to get this wrong.  I don't believe it
unreasonable to consider this sort of thing to be marked "hard hat
area".  In both directions.

On 11 February 2013 14:46, William Chan (陈智昌) <[email-masked]> wrote:
>> >> On Mon, Feb 4, 2013 at 5:47 PM, William Chan (陈智昌)
>> >> <[email-masked]>
>> >> wrote:
>> >>>
>> >>> > <[email-masked]>,
>> >>> > Ben Nive
>> >>> > n-Jenkins writes:
>> >>> >

---

From: Roberto Peon <[email-masked]>
Date: Mon, 11 Feb 2013 14:57:46 -0800

I think that the overall complexity spend is lower when we put the
complexity into prioritization mechanisms instead of stuffing heuristics
into the flow control... (because that is just ewww and ouch in so many
ways..)

-=R

On Mon, Feb 11, 2013 at 2:53 PM, Martin Thomson <[email-masked]>wrote:

> > On Mon, Feb 11, 2013 at 2:39 PM, Martin Thomson <
> [email-masked]>
> > wrote:
> >>
> >> > Let's not talk about them yet. I share your complexity concern and
> agree
> >> > we
> >> > have a forward proxy with users sharing the same HTTP/2.0 connection
> to
> >> > an
> >> > starve out others, unless we implement the vague notion of "don't
> starve
> >> > out
> >> > or that clients should have a reprioritization facility. I think this
> is
> >> > a
> >> >>> On Tue, Feb 5, 2013 at 5:34 AM, Poul-Henning Kamp <
> [email-masked]>
> >> >>> wrote:

---

From: Martin Thomson <[email-masked]>
Date: Mon, 11 Feb 2013 15:21:42 -0800

That's not at all what I had in mind, just a general re-iteration of
the basic principles:

Primarily:  Send bits based on availability and relative priorities.
Secondarily:  Advertise flow control window availability based on when
those bits are consumed (i.e. sent onwards).

On 11 February 2013 14:57, Roberto Peon <[email-masked]> wrote:
>> > On Mon, Feb 11, 2013 at 2:39 PM, Martin Thomson
>> > <[email-masked]>
>> > wrote:
>> >>
>> >> >> starve out streams.  However, this is probably a wimpy approach and
>> >> >> I
>> >> >> think
>> >> >> we can do better.
>> >> >>
>> >> >>> On Tue, Feb 5, 2013 at 5:34 AM, Poul-Henning Kamp
>> >> >>> <[email-masked]>
>> >> >>> wrote:
>> >> >>> >>would like to express but isn't prescriptive about how anyone
>> >> >>> >> uses
>> >> >>> >> or

---

From: Roberto Peon <[email-masked]>
Date: Mon, 11 Feb 2013 15:27:19 -0800

If that is what you're saying, then I definitely misunderstood. I thought
you were advocating using flow control to reduce the effective priority of
a long-running item?
-=R

On Mon, Feb 11, 2013 at 3:21 PM, Martin Thomson <[email-masked]>wrote:

> > On Mon, Feb 11, 2013 at 2:53 PM, Martin Thomson <
> >> > Theoretically possible is one thing. But the moment we get into the
> game
> >> > of
> >> > recommendations for this behavior) only use flow control (if they use
> it
> >> > at
> >> >> On 10 February 2013 14:40, William Chan (陈智昌) <[email-masked]
> >
> >> >> wrote:
> >> >> > As far as grouping, I think it's definitely a bug for the case
> where
> >> >> > you
> >> >> > starvation, I think it's debatable about whether the server should
> be
> >> >> > smart
> >> >> >> implement naively.  This isn't strictly a "bug", since the spec
> says
> >> >> >> the
> >> >> >> server can do whatever it wants with priorities, but this is
> subtle
> >> >> >> enough
> >> >> >> The bare minimum would be to simply document it and tell servers
> not
> >> >> >> to
> >> >> >> which is competing across a shared proxy backend with other
> streams;
> >> >> >> unless
> >> >> >>> sender should ultimately be in control of how it orders
> responses,
> >> >> >>> and
> >> >> >>> to rely on them, and will be forced to implement prioritization
> at
> >> >> >>> a
> >> >> >>> semantics. I am hopeful that these new semantics will not
> introduce
> >> >> >>> an
> >> >> >>> current SPDY priority levels would apply within a "group". I
> think
> >> >> >>> we

---

From: Martin Thomson <[email-masked]>
Date: Mon, 11 Feb 2013 15:35:00 -0800

My bad, I jumped from that to the next point.  That is, once you start
de-prioritizing streams, you have to have a mechanism to prevent you
from being overwhelmed by incoming packets on that stream.

Note also that the proxy can apply prioritization based on more than
just the priority field.  It can actively avoid starvation based on
more than just the priority field.

(On a random hunch, could intermediaries need flow control windows
sized for the BDP of the complete path from origin to client?)

On 11 February 2013 15:27, Roberto Peon <[email-masked]> wrote:
> you were advocating using flow control to reduce the effective priority of a
> long-running item?
> -=R
>
>
>> > On Mon, Feb 11, 2013 at 2:53 PM, Martin Thomson
>> >> > trying to carve up portions of BDP via per-stream flow control
>> >> > windows
>> >> > for
>> >> > prioritization purposes in normal operation (as opposed to just
>> >> > trying
>> >> > to
>> >> > and likely to get into performance issues due to poor
>> >> > implementations.
>> >> > As
>> >> > I've stated before, I hope most implementations (and believe we
>> >> > should
>> >> > add
>> >> >> On 10 February 2013 14:40, William Chan (陈智昌)
>> >> >> <[email-masked]>
>> >> >> wrote:
>> >> >> > need data before proceeding with many of those features, and I
>> >> >> > plan
>> >> >> > on
>> >> >> > getting data.
>> >> >> >
>> >> >> > and not allow streams to _completely_ starve other streams within
>> >> >> > a
>> >> >> > group,
>> >> >> > discussion worth having, but I'd personally classify it as
>> >> >> > separate
>> >> >> > from
>> >> >> >>> protocol bug and thus change the protocol in such a way as to
>> >> >> >>> fix
>> >> >> >>> this

---

From: Roberto Peon <[email-masked]>
Date: Mon, 11 Feb 2013 15:40:42 -0800

Gotcha, yes, though hopefully that wouldn't' happen if any
re-prioritization messages were passed back to the source of the packets!

I think that intermediaries need outgoing buffers to be BDP for the
outbound size, and incoming buffers to be BDP for the incoming path :)
That *should* sum up to the BDP of the complete path, but arguably could be
different if the proxy induces processing delay (potentially
intentionally...)
-=R

On Mon, Feb 11, 2013 at 3:35 PM, Martin Thomson <[email-masked]>wrote:

> > On Mon, Feb 11, 2013 at 3:21 PM, Martin Thomson <
> >> >> On 11 February 2013 14:46, William Chan (陈智昌) <[email-masked]
> >
> >> >> wrote:
> >> >> >> > First, I totally agree SPDY/4 prioritization changes are far
> more
> >> >> >> > reaching.
> >> >> >> > origin server. You can have many high priority short-lived
> streams
> >> >> >> > which
> >> >> >> > can
> >> >> >> > streams", which is difficult when it might be a sustained rate
> of
> >> >> >> > high
> >> >> >> >> (surfacing primarily in http/2 -> http/2 proxy situations),
> that
> >> >> >> >> many
> >> >> >> >> server
> >> >> >> >> The SPDY/4 prioritization changes are far more reaching than
> just
> >> >> >> >> fixing
> >> >> >> >> this bug.  While I like grouping as a feature, I don't believe
> it
> >> >> >> >> actually
> >> >> >> >>> semantics are generally not respected, then clients will not
> be
> >> >> >> >>> able
> >> >> >> >>> new semantics. I am arguing that because the existing
> mechanisms
> >> >> >> >>> for
> >> >> >> >>> inordinate amount of specification, as the primary idea is
> that
> >> >> >> >>> the
> >> >> >> >>> can come up with a way to define a group that will be
> relatively
> >> >> >> >>> easy
> >> >> >> >>> to spec.
> >> >> >> >>>
> >> >> >> >>> but I wanted to focus on this one first, as I believe this is
> a
> >> >> >> >>> bug
> >> >> >> >>> that we *need* to fix. The other SPDY/4 priority changes are
> of
> >> >> >> >>> a
> >> >> >> >>> performance optimization nature, and I believe they will need
> to
> >> >> >> >>> be
> >> >> >> >>> >>express the different priorities between & within groups
> that
> >> >> >> >>> >> folks
> >> >> >> >>> > Never attribute to malice what can adequately be explained
> by
> >> >> >> >>> > incompetence.
> >> >> >> >>>
> >> >> >> >>
> >> >> >> >
> >> >> >
> >> >> >
> >> >>
> >> >
> >
> >
>

---

From: Nico Williams <[email-masked]>
Date: Mon, 11 Feb 2013 17:59:44 -0600

> all, which hopefully they don't because it's hard) for maintaining

Right.  Don't duplicate the SSHv2 handbrake (Peter Gutmann's term) in HTTP/2.0.

Use percentages of BDP on the sender side.  Have the receiver send
control frames indicating the rate at which it's receiving to help
estimate BDP, or ask TCP.  But do not flow control.

Another possibility is to have the sender (or a proxy) use
per-priority TCP connections.

Nico
--

---

From: Yoav Nir <[email-masked]>
Date: Tue, 12 Feb 2013 07:13:23 +0000

> per-priority TCP connections.

I don't think that one solves the problem. A server has to consider priority as relative to the TCP connection, so that high-priority requests trump low-priority requests within the same connection, but not low-priority requests in another connection. Otherwise we have a fairness issue even without proxies.

So you're effectively creating several streams, each with all requests having the same priority. The server will then try to be fair to all connections, effectively giving the same performance to high-priority and low-priority requests.

Yoav

---

From: Nico Williams <[email-masked]>
Date: Tue, 12 Feb 2013 10:16:41 -0600

> I don't think that one solves the problem. A server has to consider priority as relative to the TCP connection, so that high-priority requests trump low-priority requests within the same connection…

Clearly with per-priority TCP connections there's no need for explicit
priority labels.  The reason for wanting multiple flows is to avoid
the situation where bulk transfers block the smaller requests (and
responses) needed for applications to remain responsive to user input.
 The moment different QoS traffic is multiplexed over one TCP
connection we need either nested flow control (bad!) or other
cooperation between the sender and receiver (hop-by-hop too) to ensure
timely delivery of non-bulk, high-priority requests.

> So you're effectively creating several streams, each with all requests having the same priority. The server will then try to be fair to all connections, effectively giving the same performance to h…

Not necessarily.  First, the small requests can get through even
though the TCP connection for bulk transfers is full enough that it'd
take much longer for the small requests to get through (possibly
because of I/O problems with bulk sinks/sources on the server side
causing flow control to kick in).  Second, the server can probably
apply application-specific [possibly heuristic] rules to prioritize
processing of some requests over others regardless of which TCP
connections they arrived over.

I'm not advocating per-priority TCP connections.  I'm specifically
arguing against SSHv2-style per-channel flow control -- a performance
disaster -- and offering and supporting alternartives.

Nico
--

---

From: Roberto Peon <[email-masked]>
Date: Tue, 12 Feb 2013 10:18:36 -0800

The problem that we have here is that the TCP API isn't sufficiently rich
to allow us to do the right thing (e.g. read bytes without allowing the
sender to send more). As a result, we have to have another level of flow
control which we'd otherwise be able to do without. Unfortunately, per
priority TCP connections don't work well for large loadbalancers where each
of these connections will likely be terminating at a different place. This
would create a difficult synchronization problem server side, full of races
and complexity, and likely quite a bit worse in complexity than getting
flow control working well.

Note that the recommendation will be that flow control be effectively
disabled unless you know what you're doing, and have a good reason (memory
pressure) to use it.

-=R

On Tue, Feb 12, 2013 at 8:16 AM, Nico Williams <[email-masked]>wrote:

> > I don't think that one solves the problem. A server has to consider
> priority as relative to the TCP connection, so that high-priority requests
> trump low-priority requests within the same connection, but not
> low-priority requests in another connection. Otherwise we have a fairness
> issue even without proxies.
>
> > So you're effectively creating several streams, each with all requests
> having the same priority. The server will then try to be fair to all
> connections, effectively giving the same performance to high-priority and
> low-priority requests.
>

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 13 Feb 2013 17:39:22 +1300

On 13/02/2013 7:18 a.m., Roberto Peon wrote:
>
> The problem that we have here is that the TCP API isn't sufficiently 
> rich to allow us to do the right thing (e.g. read bytes without 
> allowing the sender to send more). As a result, we have to have 
> another level of flow control which we'd otherwise be able to do 
> without. Unfortunately, per priority TCP connections don't work well 
> for large loadbalancers where each of these connections will likely be 
> terminating at a different place. This would create a difficult 
> synchronization problem server side, full of races and complexity, and 
> likely quite a bit worse in complexity than getting flow control 
> working well.

As opposed to HTTP/1.x where each request may have its own TCP 
connection, end up at a different endpoint, and faces identical situation?
Sure this is not a solution to *that* problem, but nothing will.
1) The load balancers very existence purpose is to *create* that problem.

2) HTTP is meant to be stateless remember, so no request depends on any 
other for semantic handling. These high priority and low-priority 
requests you speak of.

3) the frames where statefulness and priority actually matters are 
per-hop control frames. Which the endpoint needing to send is perfectly 
capable of injecting for delivery between any other frames it may have 
queued. Regardless of how or when any other party along the channel 
believes they should be delivered.

So I agree with Nico on this one. Separate TCP connections *are* an 
option. It just has to be made clear in the spec how to identify them at 
the server and how it is expected to prioritize those connections from 
any other more important ones it may be handling in parallel. Any 
endpoint screwups caused by load balancing as a result of handling two 
*independent* requests via different pathways is the responsibility of 
the server admin, site author, or network admins to sort out - not the 
HTTP protocol.

Amos

---

From: Nico Williams <[email-masked]>
Date: Wed, 13 Feb 2013 14:57:13 -0600

On Tue, Feb 12, 2013 at 12:18 PM, Roberto Peon <[email-masked]> wrote:
> The problem that we have here is that the TCP API isn't sufficiently rich to
> allow us to do the right thing (e.g. read bytes without allowing the sender
> to send more). As a result, we have to have another level of flow control

That's not necessary here.

There are two issues here:

a) flow [and congestion] control;
b) prioritization of "interactive" or "control" traffic over bulk traffic.

These are exactly the same issues that have been faced in SSHv1 and SSHv2.

TCP can handle (a), but if you multiplex traffic of different QoS over
one TCP connection you run into the issues that SSHv1 and v2 have run
into.

There's two ways to address these issues: either don't it (it ==
multiplex diff QoS traffic over the same TCP conn.) or try hard never
to write more than one BDP's worth of bulk without considering higher
priority traffic.  Determining BDP is non-trivial and it can vary, but
it's reasonable to estimate it by looking at round-trip times (it'd be
nice if TCP could expose that to apps so they don't have to measure it
redundantly!) and growing send bandwidth until receive bandwidth stops
growing -- not exactly trivial, but reasonable.

Now, in practice browsers already use multiple TCP connections to the
same server anyways, so... what's wrong with per-priority TCP
connections?  (see below)

> which we'd otherwise be able to do without. Unfortunately, per priority TCP
> connections don't work well for large loadbalancers where each of these
> connections will likely be terminating at a different place. This would
> create a difficult synchronization problem server side, full of races and
> complexity, and likely quite a bit worse in complexity than getting flow
> control working well.

I think you're saying that because of proxies it's difficult to ensure
per-priority TCP connections, but this is HTTP/2.0 we're talking
about.  We have the power to dictate that HTTP/2.0 proxies replicate
the client's per-priority TCP connection scheme.

> disabled unless you know what you're doing, and have a good reason (memory

Huh?  Are you saying "we need and will specify flow control.  It won't
work.  Therefore we'll have it off by default."  How can that help?!
I don't see how it can.

Nico
--

---

From: Roberto Peon <[email-masked]>
Date: Wed, 13 Feb 2013 13:23:18 -0800

On Wed, Feb 13, 2013 at 12:57 PM, Nico Williams <[email-masked]>wrote:

> rich to

We have more levels of prioritization than just that, but yes.

> one TCP connection you run into the issues that SSHv1 and v2 have run

Agreed -- varying QoS for packets on a single in-order stream (i.e.
connection) basically don't help, even if the network did the right thing
with them, which it may not.
Even if the network does the right thing and the bytes have arrived, TCP's
API still only lets you access the packets in-order.

> priority traffic.

QoS for packets on multiple connections also doesn't work- each entity
owning a connection sends at what it believes is its max rate, induces
packet loss, gets throttled appropriately, and then takes too make RTTs to
recover. You end up not fully utilizing the channel(s).

>  Determining BDP is non-trivial and it can vary, but
The hard part is "considering higher priority traffic" when that traffic is
being send from a different machine, as would occur in the multiple
connection case.
With a single connection, this is easy to coordinate. Agreed that
estimating BDP isn't trivial (however it is something that TCP effectively
has to do).

> the client's per-priority TCP connection scheme.

No, I'm saying that it is somewhere between difficult and impossible to
ensure that separate connections from a client end up on one machine in the
modern loadbalancer world.
>From a latency perspective, opening up the multiple connections can be a
loss as well-- it increases server load for both CPU and memory and vastly
increases the chance that you'll get a lost-packet on the SYN which takes
far longer to recover from as it requires an RTO before RTT has likely been
computed.

> work.  Therefore we'll have it off by default."  How can that help?!
Everyone will be required to implement the flow control mechanism as a
sender.
Only those people who have effective memory limitations will require its
use when receiving (since the receiver dictates policy for flow control).

-=R

---

From: Nico Williams <[email-masked]>
Date: Wed, 13 Feb 2013 15:48:28 -0600

> API still only lets you access the packets in-order.

Are we brave enough to try SCTP instead of TCP for HTTP/2.0?

I didn't think so.

:) (that should be a sad smiley, actually)

> recover. You end up not fully utilizing the channel(s).

No, no, all bulk traffic should be sent over one connection at max
rate.  Multiple bulk flows can be multiplexed safely over one TCP
connection, therefore they should be.

High priority traffic _generally_ means "non-bulk", therefore "max
rate" for non-bulk is generally much, much less than for bulk and,
therefore, non-bulk traffic can be multiplexed safely over a single
TCP connection, being careful to move to a bulk connection when a
non-bulk flow changes nature.

The sender will know what whether a message is a bulk message or not.

One complication here is that many requests will be non-bulk but their
responses will be.  I.e., you might want to write the responses to
requests on a different connection from the request!  And now you need
an XID or some such, but you probably want one anyways so that
responses can be interleaved.

(For example by analogy, if we were talking about doing this as an
SSHv2 extension we might migrate a pty stdout channel to a bulk
connection when the user does a cat(1) of a huge file.  This is much
harder in SSHv2 because we have logical octet streams for interactive,
high-priority data, but we don't have such a thing in HTTP, so this is
not a concern at all.  This is just an analogy to illustrate the
point.)

> being send from a different machine, as would occur in the multiple

Are you talking about proxies aggregating traffic from multiple
clients into one [set of] TCP connection[s] to a given server?  Sure,
but all the proxy needs is to know whether a given request (or
response) is bulk or not.

> With a single connection, this is easy to coordinate. Agreed that estimating
> BDP isn't trivial (however it is something that TCP effectively has to do).

A single connection is a bad idea.  We already use multiple
connections today in _browsers_.  Of course, for non-browser apps
multiple connections may be quite a change, but that should be a)
optional, b) acceptable anyways.

> modern loadbalancer world.

I don't think it should be difficult, much less impossible, for
HTTP/_2.0_.  What you need for this is to identify flows so their
requests/responses can be grouped.  The main thing that comes to mind
is that the load balancer needs to understand Chunked PUTs/POSTs and
get them to go to the same end server -- surely this is handled
already in HTTP/1.1 load balancers.

> far longer to recover from as it requires an RTO before RTT has likely been

Well, sure, but the sender could share one connection for multiple QoS
traffic types while the additional connections come up, and hope for
the best -- mostly it should work out.

> Only those people who have effective memory limitations will require its use
> when receiving (since the receiver dictates policy for flow control).

So this is a source quench type flow control?  (As opposed to window
size type, as in SSHv2.)  But note that the issue isn't the need to
quench fast sources from slow sinks.  The issue is that by the time
you notice that you have a source/sink bandwidth mismatch it's too
late and TCP flow control has kicked in.  Of course, the receiver can
recover by quenching the sender and then reading and buffering
whatever's left on the wire, thus freeing up bandwidth on the wire for
other sources, but the cost is lots of buffer space on the receiver,
unless you can tell the sender to resend later and then you can throw
away instead of buffer.

Nico
--

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 13 Feb 2013 14:24:33 -0800

There are a lot of emails here, and I haven't read them all. But this
discussion of browsers and multiple connections and bulk data transfers
worries me. Please remember what the browser vendors are telling you: we
want prioritization amongst HTTP requests involved in browsing, all of
which count as "interactive" and "non-bulk". Please see:
http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0568.html
http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0573.html

I don't understand how per-priority TCP connections will solve this.

On Wed, Feb 13, 2013 at 1:48 PM, Nico Williams <[email-masked]>wrote:

> estimating
> >> > which we'd otherwise be able to do without. Unfortunately, per
> priority
> >> > TCP
> use

---

From: Roberto Peon <[email-masked]>
Date: Wed, 13 Feb 2013 14:43:53 -0800

SCTP: Unfortunately not deployable due to consumer NAT interactions.

Bulk-traffic: There are a number of different levels of traffic we're
prioritizing. It isn't just 'bulk' or 'highpri'
Certain features require synchronization between control data and payload
(e.g. server push).
It is not possible to demux these without additional complexity from a
protoco standpoint.

>From an implementation standpoint: I'm already running out
of ephemeral port space. I *do Not* want to use more connections.
>From an implementation standpoint: It is often impossible to figure out
that k connections belong to a single client at all loadbalancers without
sacrificing orders of magnitude of performance. Orders of magnitude, not
just a factor of X.
There isn't even a guarantee that multiple connections will go to the same
load balancer.

Browsers have used multiple connections because of limitations in HTTP,
which we're already solving with multiplexing.

The flow control is window-update based, however a receiver can indicate to
a sender a value which effectively means: never limit yourself on flow
control.
-=R

---

From: Nico Williams <[email-masked]>
Date: Thu, 14 Feb 2013 14:26:57 -0600

> SCTP: Unfortunately not deployable due to consumer NAT interactions.

I know :(

> prioritizing. It isn't just 'bulk' or 'highpri'

I believe there's really only two or three categories of traffic:
bulk, non-bulk w/ Nagle algorithm, non-bulk w/o Nagle.  That's really
it.  If there are multiple bulk flows where it is not desirable for
one slow/stuck sink to cause all the other bulk flows to stop, then
you need a TCP connection per-bulk flow (or at least that one
possibly-slow flow).

But it's possible that we're talking about different thing.  One thing
is priority for server processing of requests.  Another is for proxies
-- here we have to start worrying about the multiplexing issues that
SSHv1 and SSHv2 have had, and since I think we ware talking about
proxies I keep coming back to these issues.

> It is not possible to demux these without additional complexity from a

I don't see why.  Can you explain in more detail?

> From an implementation standpoint: I'm already running out of ephemeral port
> space. I *do Not* want to use more connections.

It's certainly what browsers do already, and have for many years.
What's the problem?

> From an implementation standpoint: It is often impossible to figure out that
> k connections belong to a single client at all loadbalancers without

But I'm not saying you need to do that.  Nor am I implying it.

> There isn't even a guarantee that multiple connections will go to the same

All that matters is that bulk traffic be on separate TCP connections
from non-bulk.  That's one bit in the request/response headers.

> which we're already solving with multiplexing.

I strongly suspect that you're not solving them.  It's not just "oh,
let's multiplex".  You need to watch out for the issues other
protocols that multiplex different flows on single TCP flows have had.
 You haven't demonstrated a grasp of those issues.

> a sender a value which effectively means: never limit yourself on flow

That's roughly what SSHv2 does.  It's a disaster.  If you don't
believe me, do at least go ask others who have experience in this.

Nico
--

---

From: Roberto Peon <[email-masked]>
Date: Thu, 14 Feb 2013 13:42:35 -0800

> proxies I keep coming back to these issues.

Priority and flow control are separate issues.
Priorities are an expression of the order in which the browser wants/needs
resources.
Flow control is for receivers which are memory constrained to be sure that
they won't overrun memory requirements.

> I don't see why.  Can you explain in more detail?

If you want to synchronize things which are not part of a sequence, you
must do additional buffering, above and beyond what TCP may do, and you
must also include synchronization identifiers in both streams.

> It's certainly what browsers do already, and have for many years.

Browsers do this because HTTP effectively limits you to
one outstanding request per connection, which is effectively one request
per RTT per connection.
Allowing a larger concurrency per connection than 1 is what the
multiplexing does.

Browsers are most often not the ones likely to run out of ephemeral port
space-- that is a problem for proxies and servers.

> But I'm not saying you need to do that.  Nor am I implying it.

If there are multiple connections, then the cost of coordination for this
kind of loadbalancing at large sites is orders of magnitude higher.
You're telling me that you wish to use multiple connections. Unless my
statement above is a lie, it must then follow that it will cost orders of
magnitude more for the same coordination.

> from non-bulk.  That's one bit in the request/response headers.

This is not what browsers need. Browsers need to control the order in which
the server sends responses effectively enough that they can simply send
requests for resources as soon as they know about the resource.
Today they cannot, and attempt to control this by using heuristics to
decide when to send a requests (instead of simply sending it as soon as the
browser realizes it needs the resource). This causes an underutilization of
the available bandwidth.
Removal of this heuristic increases the amount of time it takes to load the
page.

>  You haven't demonstrated a grasp of those issues.

Nico, I've been working closely with browser folks for quite a while
(years) and have a good grounding in what is suboptimal today, after all,
SPDY started as a project with a server/proxy developer (me) and a browser
developer (Mike Belshe).
Will Chan and others have been continuing the work on the Chrome side, and
he has already replied to you telling you what browsers need.
I've done several presentations and talks, several of them with browser
folks which do reference the primary browser issue of when to generate
requests.

You are a smart guy and likely have things to contribute, but, please
research before making assertions about what I do or do not know.
-=R

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 15 Feb 2013 19:40:44 +1300

>> SCTP: Unfortunately not deployable due to consumer NAT interactions.

That being todays problem.  Are these NAT interactions expected to still 
be widespread into the 10-20 years future lifetime of HTTP/2. Noting 
that these NAT obstacles do not exist in some networks already, and with 
IPv6 rollout are declining in popularity over time.

ie, is it worth the effort to split the Draft by transport under-layer 
and document HTTP/2 over SCTP as well as over TCP?

> it.

I was party to some HTTP object size evaluations a few years back. There 
were at the time four 'bumps' in the average object size. Roughly:

0-64 Byte entity - common to all networks, this roughly matches:
   * the HTTP/1.1 prevalence of 0-sized 304 validation messages,
   * the 1990's habitual breaking of images into thousands of <128 byte 
tiles,
   * the 2006+ habit of using compressed PNG icons for graphic design.

4-40 KB entity (network dependent peak around 8K, 11K, or a flatish 15K-31K)
   * HTML and images on websites optimized for speed or client experience
   * CSS and early 2000's JavaScript files.

10-20 MB entity
  * MP3 and WAV audio files from that early 2000's fad on embeded sound
  * video files from YouTube and similar video sites.
  * short streams and podcasts.
  * some backup systems tarball or equivalent file transfers

0.5 - 8 GB entity
  * CD and DVD images
  * VM image transfers
  * bulk archive transfers between servers
  * streaming media

These varied between the few networks I got a good look at, and there 
were of course a lot of traffic in the gaps between but the overall 
graph looks like a declining cosine wave.

I think there are at least 3 priority types identified in the above 
sets, and I'm not sure where all the long-polling 'services' and such 
will fit in once they are identified. Perhapse they were already 
contributing but not noticed in the flood of web-page traffic?

>    If there are multiple bulk flows where it is not desirable for

Given two connections between client A and "example.com" (one for high 
priority control frames, the other for large bulk frames). If client A 
wants to abort a push it emits a control frame to stop it. Sounds 
simple, BUT...

* What if the control frame priority channel was different to the 
priority of the channel it is trying to stop? - what if those are 
different endpoints without the client A knowing? Note that when there 
is more than just client<->origin involved the client _can't_ know TCP 
level details about what the middle is doing.

* Do we let critical control frames get HOL blocked (or frame-size 
blocked at least) on the bulk channel just because they refer to that 
channels content? - if so what is the use of having priority in the 
first place and not just splitting requests over multiple connections?

* if the control frame does reach a single origin, but via two 
middleware paths... what are the middleware which never reach the STOP 
frame expected to do? what if middleware on the bulk path wants to emit 
STOP by itself? now the origin must handle control frames from multipe 
channels from independent sources.

... complexity. And not just complexity. That last case with middleware 
policing transfer size adds security aspects as well.

Lets not get buried in this ants nest.

> It's certainly what browsers do already, and have for many years.

As we middleware authors put forward in the initial arguments against 
using SPDY /w TLS as a basis for HTTP/2. Browsers are not getting 
anywhere close to their TCP resource limits. Proxies and servers face 
*massively* higher port counts than individual browsers (ratios in the 
order of >20000:1 is normal). Just because browsers can consume more 
resources does not mean the other end of the connection can spare them.

Amos

---

From: Nico Williams <[email-masked]>
Date: Mon, 18 Feb 2013 20:07:50 -0600

> Priorities are an expression of the order in which the browser wants/needs

Ah, if that's all priorities do, then there's no issue.  However, if
they relate to traffic classes (bulk vs. non-bulk) then we have the
SSH problems.

> they won't overrun memory requirements.

It's for sinks that are slow.  All receivers are memory constrained, after all.

> Browsers do this because HTTP effectively limits you to one outstanding
> request per connection, which is effectively one request per RTT per
> connection.

Pipelining allows multiple requests to be sent before responses are
written back.  The problem is that responses must be written in the
same order as the requests.  The solution to that would be to add an
XID (like basically any RPC protocol would, e.g., ONC RPC).

> Allowing a larger concurrency per connection than 1 is what the multiplexing
> does.

I suspect that's not the only reason that browsers make multiple
concurrent connections to servers.  To be fair, the SSH problems can't
the reason either.

> space-- that is a problem for proxies and servers.

Ah!  This is a good thing to point out.  But note that proxies are in
a decent position to know if SCTP can work (namely, they can cache
which servers it's worked for and which it hasn't).  So an SCTP
binding of HTTP might still be useful.

> magnitude more for the same coordination.

I'm telling you that if SSH multiplexing/flow control issues arise
then the best solution is to have two TCP connections: one for bulk,
and one for non-bulk.  Bulk/not-bulk adds a single boolean input to
the routing process.  I don't see how that raises the cost of load
balancing by one order of magnitude, much less two, not unless issues
like scarcity of ephemeral ports are the root cause (load balancers
are unlikely to have this problem, but proxies are).

> Nico, I've been working closely with browser folks for quite a while (years)
> and have a good grounding in what is suboptimal today, after all, SPDY
> started as a project with a server/proxy developer (me) and a browser
> developer (Mike Belshe).

You're right, I did not express myself very well.  I apologize.  I am
concerned with any notion of flow control layered above TCP.  I'm not
as concerned about priorities now that I understand their purpose (I
did suspect we might be talking about different things.)

Nico
--

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 18 Feb 2013 18:40:09 -0800

I'm happy to see that like what happened in the other thread, lots of the
disagreement ultimately was due to confusion over what was actually being
discussed. For prioritization, let's remember that we are discussing the
stream prioritization mechanism as described in the draft spec (
http://htmlpreview.github.com/?https://github.com/http2/http2-spec/blob/master/draft-ietf-httpbis-http2.html#StreamPriority)
which expresses the order in which the client advises the server to service
streams.

As before, I'd like to reiterate (and I believe most people have agreed
here) that we should not use receive windows to achieve prioritization.

Nico, I think the discussion of whether or not HTTP/2 stream prioritization
is useful for different traffic classes is a useful discussion to have. I
suggest starting a separate thread, so we don't incorrectly conflate it
with the stream prioritization issue I raised here.

> It's for sinks that are slow.  All receivers are memory constrained, after

Correct, although browsers typically are less concerned than servers. But
that's generally because their sinks (CPU processing) are generally faster
than the sources (network) for HTTP traffic.

>
> >> > From an implementation standpoint: I'm already running out of
> ephemeral
> >> > port

> multiplexing

As a browser vendor, I can say that the primary reason to use multiple
connections is to achieve parallelization without head of line blocking. A
downside of multiple connections is increased contention (primarily
bandwidth) which can definitely slow down page load. We currently do not do
anything smart with regard to bulk transfers (like downloads). It's
possible we could try to use TCP receive windows to control this, but I'd
rather not go there as it's difficult to get right. I know that Patrick has
expressed interest in dabbling here for Firefox, so I guess this is one of
those situations where I'm more conservative than he. I look forward to him
gathering results first :)

> binding of HTTP might still be useful.

Alternative transports may indeed be useful. But let's focus on TCP for
now, and just keep other possible transports in mind, as per the charter.

> are unlikely to have this problem, but proxies are).

I think the discussion of what mechanism to utilize to address different
traffic classes is a good one to have, but should be done in a separate
thread.

> (years)

If you have concerns about the flow control proposal, please respond to
them there.
http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0714.html is
the current thread on the topic.