Subject: WINDOW_UPDATE with streamid = 0 redundant?

From: Daniel Sommermann <[email-masked]>
Date: Tue, 11 Feb 2014 14:33:25 -0800

"Separate WINDOW_UPDATE frames are sent for the stream and connection 
level flow control windows. "

It seems that we don't usually need a separate WINDOW_UPDATE for the 
connection (with streamid = 0). When a particular stream gets processed, 
we send a WINDOW_UPDATE for that stream, but doesn't this also imply 
that the connection-level window should also be updated? When would a 
WINDOW_UPDATE for a stream NOT imply an update on the connection window?

I can see keeping around WINDOW_UPDATE with streamid = 0 for the case 
where you want to increase only the connection window size at the 
beginning of the connection, but after that setup, it seems like a waste 
of bandwidth to send these extra WINDOW_UPDATE frames for the connection 
window.

Thoughts?

---

From: Martin Thomson <[email-masked]>
Date: Tue, 11 Feb 2014 14:40:17 -0800

On 11 February 2014 14:33, Daniel Sommermann <[email-masked]> wrote:
> "Separate WINDOW_UPDATE frames are sent for the stream and connection level
> flow control windows. "
>
> connection (with streamid = 0). When a particular stream gets processed, we
> send a WINDOW_UPDATE for that stream, but doesn't this also imply that the
> connection-level window should also be updated? When would a WINDOW_UPDATE
> for a stream NOT imply an update on the connection window?
>
> I can see keeping around WINDOW_UPDATE with streamid = 0 for the case where
> you want to increase only the connection window size at the beginning of the
> connection, but after that setup, it seems like a waste of bandwidth to send
> these extra WINDOW_UPDATE frames for the connection window.
>
> Thoughts?

There are two layers of flow control windows, one per stream, and a
single connection-level window.

WINDOW_UPDATE on stream 0 increases the space advertised on the
connection-level window.  It's critical to protocol operation that
this be separately updated to the stream-level windows.

---

From: Daniel Sommermann <[email-masked]>
Date: Tue, 11 Feb 2014 14:47:42 -0800

Right, but as I understand it, the connection-level limit is meant to 
model the total buffering capability of the receiver. This amount of 
buffer space is only reduced when data on a particular stream is 
buffered. When that data is processed, we eventually send a 
WINDOW_UPDATE for that stream, but in reality, the receiver has also 
freed space in its total buffer budget by the same amount, so at some 
point, a WINDOW_UPDATE for the connection will follow for the same amount.

What I'm suggesting is that we make the connection-level WINDOW_UPDATE 
implicit with the stream-level WINDOW_UPDATE, since it fits with 
modeling the buffering capability of the remote side.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 11 Feb 2014 14:58:14 -0800

No, it is very very very important that they're separate.
The sum of the connection level window is almost always < sum(stream
windows), and this is necessary to ensure that large stream concurrencies
can be advertised.

-=R

On Tue, Feb 11, 2014 at 2:47 PM, Daniel Sommermann <[email-masked]> wrote:

> model the total buffering capability of the receiver. This amount of buffer
> space is only reduced when data on a particular stream is buffered. When
> that data is processed, we eventually send a WINDOW_UPDATE for that stream,
> but in reality, the receiver has also freed space in its total buffer
> budget by the same amount, so at some point, a WINDOW_UPDATE for the
> connection will follow for the same amount.
>
> implicit with the stream-level WINDOW_UPDATE, since it fits with modeling
> the buffering capability of the remote side.
>
>
>>> we
>>> where

---

From: Roberto Peon <[email-masked]>
Date: Tue, 11 Feb 2014 14:58:47 -0800

Sorry that could be confusing as stated, here is the same thing without
(hopefully) the confusing bit!

The connection level window is almost always < sum(stream windows), and
this is necessary to ensure that large stream concurrencies can be
advertised.

On Tue, Feb 11, 2014 at 2:58 PM, Roberto Peon <[email-masked]> wrote:

> On Tue, Feb 11, 2014 at 2:47 PM, Daniel Sommermann <[email-masked]>wrote:
>

---

From: Daniel Sommermann <[email-masked]>
Date: Tue, 11 Feb 2014 15:06:21 -0800

Ah I see, the problem is that individual streams may send WINDOW_UPDATE 
such that their size exceeds their SETTINGS_INITIAL_WINDOW_SIZE, and 
that would increase the size of the connection window above it's desired 
size unexpectedly.

If clients never sent WINDOW_UPDATEs such that the window exceeded its 
initial size I think the proposal could work, but that is common 
practice now. Thanks for the clarification.

On 02/11/2014 02:58 PM, Roberto Peon wrote:
> Sorry that could be confusing as stated, here is the same thing 
> without (hopefully) the confusing bit!
>
> The connection level window is almost always < sum(stream windows), 
> and this is necessary to ensure that large stream concurrencies can be 
> advertised.
>
>
> On Tue, Feb 11, 2014 at 2:58 PM, Roberto Peon <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     The sum of the connection level window is almost always <
>     sum(stream windows), and this is necessary to ensure that large
>     stream concurrencies can be advertised.
>
>     -=R
>
>
>     On Tue, Feb 11, 2014 at 2:47 PM, Daniel Sommermann
>     <[email-masked] <mailto:[email-masked]>> wrote:
>
>         Right, but as I understand it, the connection-level limit is
>         meant to model the total buffering capability of the receiver.
>         This amount of buffer space is only reduced when data on a
>         particular stream is buffered. When that data is processed, we
>         eventually send a WINDOW_UPDATE for that stream, but in
>         reality, the receiver has also freed space in its total buffer
>         budget by the same amount, so at some point, a WINDOW_UPDATE
>         for the connection will follow for the same amount.
>
>         What I'm suggesting is that we make the connection-level
>         WINDOW_UPDATE implicit with the stream-level WINDOW_UPDATE,
>         since it fits with modeling the buffering capability of the
>         remote side.
>
>
>             On 11 February 2014 14:33, Daniel Sommermann
>             <[email-masked] <mailto:[email-masked]>> wrote:
>

---

From: Johnny Graettinger <[email-masked]>
Date: Tue, 11 Feb 2014 18:11:23 -0500

My understanding is there may also be scenarios where a client wishes to
defer stream window updates, but not session updates. Eg if a client
accepts a pushed stream but isn't ready to consume it yet, it might want to
leave the stream stalled while updating the session window.

On Tue, Feb 11, 2014 at 6:06 PM, Daniel Sommermann <[email-masked]> wrote:

> such that their size exceeds their SETTINGS_INITIAL_WINDOW_SIZE, and that
> would increase the size of the connection window above it's desired size
> unexpectedly.
>
> initial size I think the proposal could work, but that is common practice
> now. Thanks for the clarification.
>
>

---

From: Roberto Peon <[email-masked]>
Date: Tue, 11 Feb 2014 15:12:53 -0800

And vice versa, in cases where the server behind a proxy is happy to accept
more data, but the proxy itself is under memory pressure.

-=R

On Tue, Feb 11, 2014 at 3:11 PM, Johnny Graettinger <
[email-masked]> wrote:

> On Tue, Feb 11, 2014 at 6:06 PM, Daniel Sommermann <[email-masked]>wrote:
>

---

From: Mike Bishop <[email-masked]>
Date: Tue, 11 Feb 2014 23:22:10 +0000

But implementations not only do, but should send updates beyond the initial size.  For example, the HTTP/2 equivalent to an Expect: 100-continue is to advertise a small window to the client.  Most requests will never hit the window, because they'll be GETs without bodies.  The client opens and half-closes the stream without ever sending a flow-controlled frame on it.  If small is non-zero and appropriately-sized, many POSTS can be completed in the same way.

For the case where the client wants to POST something large, the server has specified how much it can send without the server having "approved" it.  After the server sees the headers and initial portion of the body, it will send back either a WINDOW_UPDATE (i.e. continue), a HEADERS (e.g. 403, need auth before I'll take this), or a RST_STREAM (bad request).  It avoids having so much data in flight when the server expects it can make a decision without seeing all of the body and have a response racing with wasted payload.  It gives the server the ability to trade off wasted data versus a wasted RTT - but if the server decides to continue, the WINDOW_UPDATE will presumably be many times the initial stream window.

Basically, they serve different purposes - the session-level flow control prevents overwhelming your peer with more data than they can take.  The per-stream flow control prevents one stream crowding out the others within that allowed space.

From: Daniel Sommermann [mailto:[email-masked]]
Sent: Tuesday, February 11, 2014 3:06 PM
To: Roberto Peon
Cc: Martin Thomson; HTTP Working Group
Subject: Re: WINDOW_UPDATE with streamid = 0 redundant?

Ah I see, the problem is that individual streams may send WINDOW_UPDATE such that their size exceeds their SETTINGS_INITIAL_WINDOW_SIZE, and that would increase the size of the connection window above it's desired size unexpectedly.

If clients never sent WINDOW_UPDATEs such that the window exceeded its initial size I think the proposal could work, but that is common practice now. Thanks for the clarification.
On 02/11/2014 02:58 PM, Roberto Peon wrote:
Sorry that could be confusing as stated, here is the same thing without (hopefully) the confusing bit!

The connection level window is almost always < sum(stream windows), and this is necessary to ensure that large stream concurrencies can be advertised.

On Tue, Feb 11, 2014 at 2:58 PM, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:
No, it is very very very important that they're separate.
The sum of the connection level window is almost always < sum(stream windows), and this is necessary to ensure that large stream concurrencies can be advertised.

-=R

On Tue, Feb 11, 2014 at 2:47 PM, Daniel Sommermann <[email-masked]<mailto:[email-masked]>> wrote:
Right, but as I understand it, the connection-level limit is meant to model the total buffering capability of the receiver. This amount of buffer space is only reduced when data on a particular stream is buffered. When that data is processed, we eventually send a WINDOW_UPDATE for that stream, but in reality, the receiver has also freed space in its total buffer budget by the same amount, so at some point, a WINDOW_UPDATE for the connection will follow for the same amount.

What I'm suggesting is that we make the connection-level WINDOW_UPDATE implicit with the stream-level WINDOW_UPDATE, since it fits with modeling the buffering capability of the remote side.

On 02/11/2014 02:40 PM, Martin Thomson wrote:
On 11 February 2014 14:33, Daniel Sommermann <[email-masked]<mailto:[email-masked]>> wrote:
"Separate WINDOW_UPDATE frames are sent for the stream and connection level
flow control windows. "

Thoughts?

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 11 Feb 2014 15:32:51 -0800

On Tue, Feb 11, 2014 at 3:22 PM, Mike Bishop
<[email-masked]> wrote:
> But implementations not only do, but should send updates beyond the initial
> size.  For example, the HTTP/2 equivalent to an Expect: 100-continue is to
> advertise a small window to the client.  Most requests will never hit the
> window, because they'll be GETs without bodies.  The client opens and
> half-closes the stream without ever sending a flow-controlled frame on it.
> If small is non-zero and appropriately-sized, many POSTS can be completed in
> the same way.
>
>
>
> For the case where the client wants to POST something large, the server has
> specified how much it can send without the server having "approved" it.
> After the server sees the headers and initial portion of the body, it will
> send back either a WINDOW_UPDATE (i.e. continue), a HEADERS (e.g. 403, need
> auth before I'll take this), or a RST_STREAM (bad request).  It avoids
> having so much data in flight when the server expects it can make a decision
> without seeing all of the body and have a response racing with wasted
> payload.  It gives the server the ability to trade off wasted data versus a
> wasted RTT - but if the server decides to continue, the WINDOW_UPDATE will
> presumably be many times the initial stream window.
>
>
>
> Basically, they serve different purposes - the session-level flow control
> prevents overwhelming your peer with more data than they can take.  The
> per-stream flow control prevents one stream crowding out the others within
> that allowed space.

^^^ This nails it.

> Ah I see, the problem is that individual streams may send WINDOW_UPDATE such
> that their size exceeds their SETTINGS_INITIAL_WINDOW_SIZE, and that would
> increase the size of the connection window above it's desired size
> unexpectedly.
>
> The connection level window is almost always < sum(stream windows), and this
> is necessary to ensure that large stream concurrencies can be advertised.
>
>
>
> Right, but as I understand it, the connection-level limit is meant to model
> the total buffering capability of the receiver. This amount of buffer space
> is only reduced when data on a particular stream is buffered. When that data
> is processed, we eventually send a WINDOW_UPDATE for that stream, but in
> reality, the receiver has also freed space in its total buffer budget by the
> same amount, so at some point, a WINDOW_UPDATE for the connection will
> follow for the same amount.
>

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 12 Feb 2014 12:46:00 +1300

> modeling the buffering capability of the remote side.

That does not follow.

Consider a proxy with multiple clients. The server connection window is 
unrelated to the windows for any given client, but interdependent.

1) Sending per-stream WINDOW_UPDATE of 0 when the client connection that 
stream is going to runs out of buffer space should not be allowed to 
starve all other clients by implying 0 on the connection buffer.

2) Sending a per-stream WINDOW_UPDATE of sum(client free buffer, server 
free buffer) allows the server to flood both the client and server 
connection buffers on the proxy with bytes for that one stream. 
Effectively blocking all other multiplexed traffic if that client 
happens to be slow. AND, implying false information about available the 
server connection buffer for other clients streams.

3) Sending a per-stream WINDOW_UPDATE of sum(client free buffer, server 
free buffer) along with a separate per-connection WINDOW_UPDATE of N 
server free buffer bytes allows recipients to know how much of the 
per-stream window is specific for the receiving hop buffer and to manage 
its sending of those bytes across streams.

Amos