Subject: BLOCKED frame specification

From: Hasan Khalil <[email-masked]>
Date: Mon, 07 Apr 2014 21:43:33 +0000

Here's at least a starting point. Comments welcome.

6.x BLOCKED

The BLOCKED frame (type=0xPLACEHOLDER) conveys flow control and data
availability status useful in tuning and debugging flow control.

The BLOCKED frame contains no payload and does not define any flags.
Semantically the BLOCKED frame is used to indicate to a remote endpoint
that data subject to flow control was available to be sent, and that the
underlying transport was ready to send said data, but was prohibited from
sending said data due to either stream-level or connection-level flow
control windows being nonpositive. The specific flow control window
prohibiting the sending of said data is indicated in the stream id of the
BLOCKED frame. In the stream-level flow control blockage case, the frame's
stream identifier indicates the affected stream; otherwise the value "0"
indicates connection-level flow control blockage.

Endpoints MUST NOT send a successive BLOCKED frame on a given stream id
(including "0") unless the associated flow control window became positive
since the last time that a BLOCKED frame for that flow control window was
sent. This can happen only as a result of receiving a WINDOW_UPDATE frame
or appropriate SETTINGS frame increasing said flow control window. (TODO:
Add reference to appropriate sections of flow control spec?) (TODO: Add
GOAWAY-ENHANCE_YOUR_CALM usage here as appropriate behavior in this case?)

BLOCKED can be sent by a peer that has sent a frame bearing the END_STREAM
flag. This means that a receiver could receive a BLOCKED frame on a "half
closed (remote)" or "closed" stream. A receiver MUST NOT treat this as an
error, see Section 5.1.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Apr 2014 09:59:11 +1000

Thanks, Hasan.

What do people think?

My inclination at this point is to include this in the implementation draft, but mark it AT RISK -- i.e., we might pull it out before finalising the spec, based upon whether it turns out to be useful in interop.

Cheers,

On 8 Apr 2014, at 7:43 am, Hasan Khalil <[email-masked]> wrote:

> The BLOCKED frame (type=0xPLACEHOLDER) conveys flow control and data availability status useful in tuning and debugging flow control.
> 
> The BLOCKED frame contains no payload and does not define any flags. Semantically the BLOCKED frame is used to indicate to a remote endpoint that data subject to flow control was available to be sent, and that the underlying transport was ready to send said data, but was prohibited from sending said data due to either stream-level or connection-level flow control windows being nonpositive. The specific flow control window prohibiting the sending of said data is indicated in the stream id of the BLOCKED frame. In the stream-level flow control blockage case, the frame's stream identifier indicates the affected stream; otherwise the value "0" indicates connection-level flow control blockage.
> 
> Endpoints MUST NOT send a successive BLOCKED frame on a given stream id (including "0") unless the associated flow control window became positive since the last time that a BLOCKED frame for that flow control window was sent. This can happen only as a result of receiving a WINDOW_UPDATE frame or appropriate SETTINGS frame increasing said flow control window. (TODO: Add reference to appropriate sections of flow control spec?) (TODO: Add GOAWAY-ENHANCE_YOUR_CALM usage here as appropriate behavior in this case?)
> 
> BLOCKED can be sent by a peer that has sent a frame bearing the END_STREAM flag. This means that a receiver could receive a BLOCKED frame on a "half closed (remote)" or "closed" stream. A receiver MUST NOT treat this as an error, see Section 5.1.
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: Yoav Nir <[email-masked]>
Date: Tue, 8 Apr 2014 22:02:10 +0300

It looks good to me, except I would replace the word “debugging” with the word “troubleshooting”.  The former is usually associated with implementation development, whereas the latter is associated with figuring out why my download keeps breaking up.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 8 Apr 2014 14:15:30 -0700

> It looks good to me, except I would replace the word “debugging” with the word “troubleshooting”.  The former is usually associated with implementation development, whereas the latter is associated…

Can I infer from this response that you want this feature Yoav?

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 8 Apr 2014 23:28:03 +0200 (CEST)

On Mon, 7 Apr 2014, Hasan Khalil wrote:

> Semantically the BLOCKED frame is used to indicate to a remote endpoint that 
> data subject to flow control was available to be sent, and that the 
> control windows being nonpositive.

Maybe it should be stressed that this flow control is really the http2 flow 
control (as described in section 5.2) and no other flow that may be involved 
in a stack. A server could get tempted to send a "BLOCKED" to the client to 
signal that it could've done something better (beyond http2) that would've 
gotten it the data that it now didn't get.

I'm not convinced this frame will do much good. I'm neutral about it.

-- 

  / daniel.haxx.se

---

From: Daniel Sommermann <[email-masked]>
Date: Tue, 8 Apr 2014 14:51:25 -0700

I'm not a huge fan of this proposal as it seems to add complexity for 
not much gain. It seems debug data in the GOAWAY frame or an extra 
RST_STREAM code (e.g. BLOCKED_FLOW_CONTROL_TIMEOUT) would serve well 
enough here.

This proposal may also encourage bad protocol implementation. For 
instance, a broken flow control implementation might be patched to 
"work" by automatically sending WINDOW_UPDATE frames whenever a BLOCKED 
frame is received, despite internal state problems.

> Endpoints MUST NOT send a successive BLOCKED frame on a given stream 
> id (including "0") unless the associated flow control window became 
> positive since the last time that a BLOCKED frame for that flow 
> control window was sent. This can happen only as a result of receiving 
> a WINDOW_UPDATE frame or appropriate SETTINGS frame increasing said 
> flow control window. (TODO: Add reference to appropriate sections of 
> flow control spec?) (TODO: Add GOAWAY-ENHANCE_YOUR_CALM usage here as 
> appropriate behavior in this case?)
This requires implementations to track more state. Is there any reason 
that this is a MUST NOT? It seems like SHOULD is strong enough. How 
servers respond to many BLOCKED frames should be left flexible. For 
instance, a pathological sequence of BLOCKED / WINDOW_UPDATE / BLOCKED / 
... etc may warrant an ENHANCE_YOUR_CALM as well.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 9 Apr 2014 11:47:51 +0300

> Can I infer from this response that you want this feature Yoav?

I want the information that it conveys, and this seems like a workable way of providing it.  I don’t think a RST_STREAM code or GOAWAY data can replace this, because they will only occur after a timeout. The BLOCKED frame can be sent after a much shorter time (1 RTT?) or even immediately.

On second thought, “debugging” is probably more important, because correct implementations should never reach this state.

Yoav

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Apr 2014 16:53:29 +0800

> On second thought, “debugging” is probably more important, because correct implementations should never reach this state.

Then why doesn’t PING suffice? You can send it at any time and there’s space for an implementation-defined payload value.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 9 Apr 2014 11:59:34 +0300

> Then why doesn’t PING suffice? You can send it at any time and there’s space for an implementation-defined payload value.

That only tells me if the connection is working, not if the resources are not coming because of some server issue (like trouble connecting to a back-end database) or because of flow control.

---

From: David Krauss <[email-masked]>
Date: Wed, 9 Apr 2014 17:02:24 +0800

> That only tells me if the connection is working, not if the resources are not coming because of some server issue (like trouble connecting to a back-end database) or because of flow control.

Use the implementation-defined payload value. It’s not portable but you don’t need portability for debugging.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 9 Apr 2014 12:17:02 +0300

> Use the implementation-defined payload value. It’s not portable but you don’t need portability for debugging.

Looking at section 6.7 of -11 I don’t see any implementation-defined payload value, just some opaque data that the receiver should copy into its response. 

Even if there was, I would be debugging my implementation against some other implementation. Not against my own.

Yoav

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 9 Apr 2014 19:28:05 +1000

> Then why doesn’t PING suffice? You can send it at any time and there’s space for an implementation-defined payload value.

My understanding from the discussion in London was that the folks proposing this want to see an interoperable way to do this, rather than ad hoc, implementation-specific ways to do it.

The observation that Hasan (or Roberto; forget which) made was that flow control debugging was possible between Chrome and Google because they're relatively tightly bound, and they can coordinate things like that. If it were (insert client here) and (some random Web site), debugging gets a lot harder.

(again, I'm just reiterating what I understood to be the argument for this expressed F2F in London)

Cheers,

---

From: Roberto Peon <[email-masked]>
Date: Wed, 9 Apr 2014 02:30:23 -0700

imho, this is useful for tuning, not just debugging.

And we've found that tuning is annoyingly difficult to get right.

Obviously I want this, in case that wasn't common knowledge already :)
-=R

On Wed, Apr 9, 2014 at 2:17 AM, Yoav Nir <[email-masked]> wrote:

> > On 2014-04-09, at 4:59 PM, Yoav Nir <[email-masked]> wrote:
> >
> >> That only tells me if the connection is working, not if the resources
> are not coming because of some server issue (like trouble connecting to a
> back-end database) or because of flow control.
> >
> > Use the implementation-defined payload value. It's not portable but you
> don't need portability for debugging.
>
> Looking at section 6.7 of -11 I don't see any implementation-defined
> payload value, just some opaque data that the receiver should copy into its
> response.
>
> Even if there was, I would be debugging my implementation against some
> other implementation. Not against my own.
>
> Yoav
>
>
>

---

From: Rob Trace <[email-masked]>
Date: Wed, 9 Apr 2014 20:38:06 +0000

The BLOCKED frame is not something we are interested in and would rather not add another late feature to the spec.  The standard is already slipping the schedule and we need to stop adding to the spec.

Thanks!!

-Rob

From: Roberto Peon [mailto:[email-masked]]
Sent: Wednesday, April 9, 2014 2:30 AM
To: Yoav Nir
Cc: David Krauss; HTTP Working Group
Subject: Re: BLOCKED frame specification

On Wed, Apr 9, 2014 at 2:17 AM, Yoav Nir <[email-masked]<mailto:[email-masked]>> wrote:

On Apr 9, 2014, at 12:02 PM, David Krauss <[email-masked]<mailto:[email-masked]>> wrote:

>
> On 2014-04-09, at 4:59 PM, Yoav Nir <[email-masked]<mailto:[email-masked]>> wrote:
>
> Use the implementation-defined payload value. It's not portable but you don't need portability for debugging.
Looking at section 6.7 of -11 I don't see any implementation-defined payload value, just some opaque data that the receiver should copy into its response.

Yoav

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Apr 2014 10:18:02 +0200

On 2014-04-09 22:38, Rob Trace wrote:
> The BLOCKED frame is not something we are interested in and would rather
> not add another late feature to the spec.  The standard is already
> slipping the schedule and we need to stop adding to the spec.
>
> Thanks!!
>
> -Rob
> ...

1) I think it's safe to say that the WG is undecided about BLOCKED. Some 
people think it's important to have. Others don't think so and want to 
move on. One way to deal with this is to write it down quickly, put it 
into the draft, and remove it later on if problems come up.

2) Schedule... Well. This is an IETF activity. The schedule almost 
always slips. No, this isn't good, but it's a matter of fact, and it's 
better to be prepared for it. Things will take time; we need to pass two 
Last Calls, and get the IESG approval. The charter says we do WG LC this 
month, and we're (I believe) going to be at least two months late with 
that. If we add these two months (and that's the optimistic way to look 
at things) to the remaining milestones, we'll end up with IESG approval 
in January 2015. So everybody should be prepared not to ship any 
products with HTTP/2 turned on by default before early next year.

Best regards, Julian

---

From: Nicholas Hurley <[email-masked]>
Date: Thu, 10 Apr 2014 17:13:16 -0700

I am not particularly interested in implementing BLOCKED, whether it holds
up the schedule or not.

On Wed, Apr 9, 2014 at 1:38 PM, Rob Trace <[email-masked]> wrote:

> not add another late feature to the spec.  The standard is already slipping
> the schedule and we need to stop adding to the spec.
>
>
>
> Thanks!!
>
>
>
> -Rob
>
>
>
> *From:* Roberto Peon [mailto:[email-masked]]
> *Sent:* Wednesday, April 9, 2014 2:30 AM
> *To:* Yoav Nir
> *Cc:* David Krauss; HTTP Working Group
> *Subject:* Re: BLOCKED frame specification
>
>
>

-- 
Peace,
  -Nick

---

From: Rob Trace <[email-masked]>
Date: Fri, 11 Apr 2014 18:12:07 +0000

On 2014-04-10 1:18, Julian Reschke wrote:
 > 2) Schedule... Well. This is an IETF activity. The schedule almost always 
> slips. No, this isn't good, but it's a matter of fact, and it's better to be 
> prepared for it. Things will take time; we need to pass two Last Calls, 
> and get the IESG  > approval. The charter says we do WG LC this 
> at things) to the remaining milestones,  > we'll end up with IESG 
> approval in January 2015. So everybody should be prepared not to 
> ship any products with HTTP/2 turned on by default before early next year.
> 
> Best regards, Julian

I agree that schedules sometimes slip, but my point was that if we want to finish anytime soon, we need to stop adding new features and bring focus to closing on the open issues.  There is already some discussion about HTTP/3 on this list, and we need to ask ourselves what new things are really important for this version vs. a future one.  Continuing to add new functionality will risk delaying much further than early next year.

Rob

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 15 Apr 2014 18:06:40 -0700

It'd be interesting in hearing why not. Do you think it's net-negative, or
is it a question of time/prioritization? And would you oppose specification
even if you didn't implement?

As Roberto points out, this is more about tuning than just debugging. If
things are outright hanging, we can probably debug that. But if performance
is just crappy, it is often non-obvious to folks how to debug that. It's my
hope that a BLOCKED frame would allow the common webservers to track stats
for BLOCKED frames to make it easier to flag these performance tuning
issues, rather than having to manually dive into traces, which I don't
expect most deployments to do. I may very well be overly-concerned here,
but I fear that many will enable HTTP/2 with flow control, and it will
"work", but be slow, and people will blame HTTP/2 and not adopt. That would
be unfortunate.

On Thu, Apr 10, 2014 at 5:13 PM, Nicholas Hurley <[email-masked]>wrote:

>>  The BLOCKED frame is not something we are interested in and would
>> rather not add another late feature to the spec.  The standard is already

---

From: David Krauss <[email-masked]>
Date: Wed, 16 Apr 2014 18:00:24 +0800

What about working within the PING frame, and making a gentleman’s agreement? It’s a conforming extension anyway:

1. If a stream seems to be blocked, send a PING with the first 4 bytes being the stream ID and next 4 bytes indicating some CHECK_BLOCKED opcode constant.

2. If the stream is indeed blocked, the server sends a PING with the stream ID, a CONFIRM_BLOCKED opcode, and the ACK bit already set.

3. In any case, if the connection is OK (which is likely in question, too), the server sends a matching reply PING. If this is received before any CONFIRM_BLOCKED, the query returns not-blocked.

An additional CONFIRM_NONBLOCKED opcode would allow distinguishing a server without the extension from a non-blocked stream, although I imagine implementations would be susceptible to false NONBLOCKED results.

For what it’s worth, I don’t see what’s wrong with being more specific about PING payloads. If it’s too much to standardize this, perhaps make some kind of registry as there is for methods.

On 2014–04–16, at 9:06 AM, William Chan (陈智昌) <[email-masked]> wrote:

> It'd be interesting in hearing why not. Do you think it's net-negative, or is it a question of time/prioritization? And would you oppose specification even if you didn't implement?
> 
> As Roberto points out, this is more about tuning than just debugging. If things are outright hanging, we can probably debug that. But if performance is just crappy, it is often non-obvious to folks how to debug that. It's my hope that a BLOCKED frame would allow the common webservers to track stats for BLOCKED frames to make it easier to flag these performance tuning issues, rather than having to manually dive into traces, which I don't expect most deployments to do. I may very well be overly-concerned here, but I fear that many will enable HTTP/2 with flow control, and it will "work", but be slow, and people will blame HTTP/2 and not adopt. That would be unfortunate.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 16 Apr 2014 03:08:16 -0700

Bleh.
We have address space in the frame opcodes, lets use that instead of adding
new semantics to things which are unstructured.
The only bit of information that is needed is when the remote end is not
blocked (confirming that it was blocked intentionally/whatever seems like a
waste). That is enough for tuning and debugging.
-=R

On Wed, Apr 16, 2014 at 3:00 AM, David Krauss <[email-masked]> wrote:

> What about working within the PING frame, and making a gentleman’s
> agreement? It’s a conforming extension anyway:
>
> 1. If a stream seems to be blocked, send a PING with the first 4 bytes
> being the stream ID and next 4 bytes indicating some CHECK_BLOCKED opcode
> constant.
>
> 2. If the stream is indeed blocked, the server sends a PING with the
> stream ID, a CONFIRM_BLOCKED opcode, and the ACK bit already set.
>
> 3. In any case, if the connection is OK (which is likely in question,
> too), the server sends a matching reply PING. If this is received before
> any CONFIRM_BLOCKED, the query returns not-blocked.
>
> An additional CONFIRM_NONBLOCKED opcode would allow distinguishing a
> server without the extension from a non-blocked stream, although I imagine
> implementations would be susceptible to false NONBLOCKED results.
>
> For what it’s worth, I don’t see what’s wrong with being more specific
> about PING payloads. If it’s too much to standardize this, perhaps make
> some kind of registry as there is for methods.
>
>
> > It'd be interesting in hearing why not. Do you think it's net-negative,
> or is it a question of time/prioritization? And would you oppose
> specification even if you didn't implement?
> >

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 17 Apr 2014 01:19:01 +1200

> new semantics to things which are unstructured.

Indeed. Since the editors or WG decided that frames shall not be
extensible there are opcodes to spare in abundance. Whereas ping codes
are potentially open to extensions still by other drafts.

> waste). That is enough for tuning and debugging.

Confirming that it is not-blocked could be as simple as sending the next
frame for the stream in response to the BLOCKED query. Or perhapse an
empty non-final DATA frame if one wishes.

/2c

Amos

---

From: Johnny Graettinger <[email-masked]>
Date: Wed, 16 Apr 2014 09:43:38 -0400

I'm not sure what is meant by a BLOCKED "query". This is (and IMO should
be) a unidirectional rather than query/response pattern.

I hesitate to suggest this, as I think Hasan's proposal is strictly better,
but an alternative might be to send a single empty DATA frame under the
blocked condition. This has the advantage of compatibility with the current
spec: receivers must handle empty DATA frames anyway, and can choose to
ignore the semantics. However it loses the explicitness of BLOCKED about
whether it's stream or session flow control that's the culprit; the
receiver would need to make a guess.

-johnny

---

From: David Krauss <[email-masked]>
Date: Wed, 16 Apr 2014 22:45:36 +0800

On 2014–04–16, at 9:43 PM, Johnny Graettinger <[email-masked]> wrote:

> I'm not sure what is meant by a BLOCKED "query". This is (and IMO should be) a unidirectional rather than query/response pattern.

Is there really a point to notifying of blockage by default? Every dependent stream is blocked as long as its dependency (or another dependency up the hierarchy) is unblocked. How often does a client need to be reminded which streams remain blocked?

The interesting information is whether something has been unblocked by the expected time. What is to be debugged is avoiding excessive dependencies (on the client side) and correct dependency resolution (on the server side).

> I hesitate to suggest this, as I think Hasan's proposal is strictly better, but an alternative might be to send a single empty DATA frame under the blocked condition. This has the advantage of compatibility with the current spec: receivers must handle empty DATA frames anyway, and can choose to ignore the semantics. However it loses the explicitness of BLOCKED about whether it's stream or session flow control that's the culprit; the receiver would need to make a guess.

An empty DATA frame will simply be coalesced (ignored) by the first intermediary. An empty segment would get through, but segmentation is reserved to the application.

---

From: Johnny Graettinger <[email-masked]>
Date: Wed, 16 Apr 2014 11:07:52 -0400

<resend from correct address>

On Wed, Apr 16, 2014 at 10:45 AM, David Krauss <[email-masked]> wrote:

> Is there really a point to notifying of blockage by default? Every
> dependent stream is blocked as long as its dependency (or another
> dependency up the hierarchy) is unblocked. How often does a client need to
> be reminded which streams remain blocked?
>
> The interesting information is whether something has been unblocked by the
> expected time. What is to be debugged is avoiding excessive dependencies
> (on the client side) and correct dependency resolution (on the server side).

I think there's a misunderstanding here. Priorities are about the relative
ordering of multiple *ready-to-write* streams. A sender can and certainly
should send a dependent stream frame, if that's what's available.

>  > I hesitate to suggest this, as I think Hasan's proposal is strictly
> better, but an alternative might be to send a single empty DATA frame under
> the blocked condition. This has the advantage of compatibility with the
> current spec: receivers must handle empty DATA frames anyway, and can
> choose to ignore the semantics. However it loses the explicitness of
> BLOCKED about whether it's stream or session flow control that's the
> culprit; the receiver would need to make a guess.
>
> An empty DATA frame will simply be coalesced (ignored) by the first
> intermediary.
>

That's fine. Flow control is also hop-by-hop.

cheers,
-johnny

---

From: David Krauss <[email-masked]>
Date: Wed, 16 Apr 2014 23:35:55 +0800

On 2014–04–16, at 11:07 PM, Johnny Graettinger <[email-masked]> wrote:

> I think there's a misunderstanding here. Priorities are about the relative ordering of multiple ready-to-write streams. A sender can and certainly should send a dependent stream frame, if that's what's available.

I didn’t say anything about priorities. A sender should only send a dependent stream frame if all its dependencies are blocked. Otherwise, the dependent stream is blocked.

Can you give an example of when a unidirectional BLOCKED frame would be sent?

> An empty DATA frame will simply be coalesced (ignored) by the first intermediary. 
> 

It’s not fine because such a debugging/tuning facility stops working if any ISP, load balancer, or firewall employs a proxy.

What is it you intend to accomplish? There may be a misunderstanding.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 16 Apr 2014 09:50:16 -0700

Flow control is hop by hop, not end to end.
On Apr 16, 2014 8:39 AM, "David Krauss" <[email-masked]> wrote:

> I didn’t say anything about priorities. A sender should only send a
> dependent stream frame if all its dependencies are blocked. Otherwise, the
> dependent stream is blocked.
>
> It’s not fine because such a debugging/tuning facility stops working if
> any ISP, load balancer, or firewall employs a proxy.
>

---

From: Roberto Peon <[email-masked]>
Date: Wed, 16 Apr 2014 10:02:22 -0700

Sorry, hit submit by accident :)

Flow control is hop by hop, not end to end, and so all we care about is
detecting the hop-by-hop flow control issues.
Specifically, we care about detecting when the remote side could not send
because its flow control window was zero.
-=R

On Wed, Apr 16, 2014 at 9:50 AM, Roberto Peon <[email-masked]> wrote:

>> I think there's a misunderstanding here. Priorities are about the
>> relative ordering of multiple *ready-to-write* streams. A sender can and
>> certainly should send a dependent stream frame, if that's what's available.
>>
>>

---

From: Nicholas Hurley <[email-masked]>
Date: Wed, 16 Apr 2014 10:51:11 -0700

For my part, it's just that I'm not particularly convinced of the
usefulness of the frame (not to say that I can't be convinced otherwise).
I'm not opposed to it being specified (assuming we don't hold up the rest
of the spec... this is very low on the "nice-to-have" list), nor do I think
it's "bad" (for some value of bad), I'm just not convinced yet.

On Tue, Apr 15, 2014 at 6:06 PM, William Chan (陈智昌)
<[email-masked]>wrote:

>> I am not particularly interested in implementing BLOCKED, whether it
>> holds up the schedule or not.
>>
>>
>> On Wed, Apr 9, 2014 at 1:38 PM, Rob Trace <[email-masked]>wrote:
>>
>>> > Use the implementation-defined payload value. It’s not portable but
>>> you don’t need portability for debugging.
>>>
>>> Looking at section 6.7 of -11 I don’t see any implementation-defined

-- 
Peace,
  -Nick

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 16 Apr 2014 15:12:28 -0700

OK, that seems quite reasonable to me. Thanks for the explanation. I can
definitely understand being skeptical of the usefulness of this frame. My
argument is definitely based on a vague fear that is difficult to quantify.
This is again one of those things that I would hope that we would be able
to demonstrate the utility of in practice before having to make a decision
of whether or not it goes into the spec.

On Wed, Apr 16, 2014 at 10:51 AM, Nicholas Hurley <[email-masked]>wrote:

> On Tue, Apr 15, 2014 at 6:06 PM, William Chan (陈智昌) <[email-masked]
> > wrote:
>
>>>> >> That only tells me if the connection is working, not if the
>>>> resources are not coming because of some server issue (like trouble
>>>> connecting to a back-end database) or because of flow control.
>>>> >

---

From: Hasan Khalil <[email-masked]>
Date: Wed, 16 Apr 2014 22:37:25 +0000

> of whether or not it goes into the spec.

...Which is why I'm 100% with Mark on putting this into the next
implementation draft and then getting opinions on its usefulness rather
than discussing in hypotheticals.

>> On Tue, Apr 15, 2014 at 6:06 PM, William Chan (陈智昌) <
>> [email-masked]> wrote:
>>

---

From: David Krauss <[email-masked]>
Date: Thu, 17 Apr 2014 09:27:14 +0800

On 2014–04–17, at 1:02 AM, Roberto Peon <[email-masked]> wrote:

> Flow control is hop by hop, not end to end, and so all we care about is detecting the hop-by-hop flow control issues.
> Specifically, we care about detecting when the remote side could not send because its flow control window was zero. 

But how do you know that the first hop is the one with blockage? The first rule of debugging is that a problem may be anywhere. If you’re testing interoperability with an external site, you certainly don’t know what’s on their end…

Apologies, I was thinking of blockage due to the dependency hierarchy, not flow control. Flow control stalls should frankly never happen. Flow control is supposed to improve reliability, and if it has bugs then the protocol is too complicated. When was the last time TCP flow control stalled an application? Anyway, any blockage-diagnosis facility would probably be more useful if it covered all possible root causes.

I’ll try and propose a flow control simplification today, which would outlaw throttling after a stream is in progress but still allow 100 CONTINUE emulation.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 16 Apr 2014 18:37:26 -0700

On Wed, Apr 16, 2014 at 6:27 PM, David Krauss <[email-masked]> wrote:

>
> On 2014-04-17, at 1:02 AM, Roberto Peon <[email-masked]> wrote:
>
> > Specifically, we care about detecting when the remote side could not
> send because its flow control window was zero.
>
> But how do you know that the first hop is the one with blockage? The first
> rule of debugging is that a problem may be anywhere. If you're testing
> interoperability with an external site, you certainly don't know what's on
> their end...
>
> Apologies, I was thinking of blockage due to the dependency hierarchy, not
> flow control. Flow control stalls should frankly never happen. Flow control
> is supposed to improve reliability, and if it has bugs then the protocol is
> too complicated. When was the last time TCP flow control stalled an
> application? Anyway, any blockage-diagnosis facility would probably be more
> useful if it covered all possible root causes.
>

TCP flow control often stalls applications. Especially when networks strip
out window scaling (this definitely happens, but I unfortunately have no
public data to share here). And then there's the historical issue with
Linux receive windows being too small by default:
https://www.belshe.com/2010/11/19/linux-client-tcp-stack-slower-than-windows/
.

>
> I'll try and propose a flow control simplification today, which would
> outlaw throttling after a stream is in progress but still allow 100
> CONTINUE emulation.
>
>
>

---

From: David Krauss <[email-masked]>
Date: Thu, 17 Apr 2014 09:48:12 +0800

On 2014–04–17, at 9:37 AM, William Chan (陈智昌) <[email-masked]> wrote:

> TCP flow control often stalls applications. Especially when networks strip out window scaling (this definitely happens, but I unfortunately have no public data to share here). And then there's the historical issue with Linux receive windows being too small by default: https://www.belshe.com/2010/11/19/linux-client-tcp-stack-slower-than-windows/.

That’s gonna make it slow, not stalled. I gather that the condition of present concern is when a client accidentally allows the window of a particular stream to go to zero, due to intentionally failing to send receipts as a throttle control. The client screwed up the accounting and gets confused.

However it can also occur upstream by an intermediary that fails to anticipate all the corner cases of such client-initiated throttling to prevent its intermediate windows from glitching to zero. (At least, I anticipate this, I haven’t yet worked through a blow-by-blow example.)

The solution is not to use flow control for relative throttling but priority instead. Don’t play with fire!

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 16 Apr 2014 18:50:28 -0700

OK, just to be clear, I'm more worried about slowness/tuning than outright
hangs. If things hang, people are going to file bugs and maintainers of
major software projects are going to fix those issues. What's more
worrisome is non-obvious issues like performance tuning.

On Wed, Apr 16, 2014 at 6:48 PM, David Krauss <[email-masked]> wrote:

> That’s gonna make it slow, not stalled. I gather that the condition of
> present concern is when a client accidentally allows the window of a
> particular stream to go to zero, due to intentionally failing to send
> receipts as a throttle control. The client screwed up the accounting and
> gets confused.
>
> However it can also occur upstream by an intermediary that fails to
> anticipate all the corner cases of such client-initiated throttling to
> prevent its intermediate windows from glitching to zero. (At least, I
> anticipate this, I haven’t yet worked through a blow-by-blow example.)
>

---

From: Yoav Nir <[email-masked]>
Date: Thu, 17 Apr 2014 11:49:26 +0300

> That’s gonna make it slow, not stalled. I gather that the condition of present concern is when a client accidentally allows the window of a particular stream to go to zero, due to intentionally fai…

It’s not necessarily a hang. It’s also possible that the client is sending WINDOW_UPDATE frames, just not sending them soon enough. Suppose the RTT is 0.5 seconds and bandwidth is 10 Mbps. If the client lets the window get down to 500 KB before sending the WINDOW_UPDATE, there could be 2.5 Mb (313 KB) en route, and within the 0.25 second it takes the WINDOW_UPDATE to reach the server, the window will empty. 

That’s the kind of thing a BLOCKED frame might help detect.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Apr 2014 12:05:46 -0700

Integrating a paraphrased version now.  A few observations below.
(Net result: even less text :)

On 7 April 2014 14:43, Hasan Khalil <[email-masked]> wrote:
> sent. This can happen only as a result of receiving a WINDOW_UPDATE frame or
> appropriate SETTINGS frame increasing said flow control window. (TODO: Add
> reference to appropriate sections of flow control spec?) (TODO: Add

During normal operation, it's going to be difficult to determine when
a WINDOW_UPDATE was seen and acted upon by a peer with any real
certainty.  SETTINGS at least has an acknowledgement.  I'm not going
to add anything for this, though I will make a note of this in the DoS
section.

> closed (remote)" or "closed" stream. A receiver MUST NOT treat this as an

This doesn't make sense.  You only want to send BLOCKED when you might
normally send DATA.