Subject: Question about RFC7540 (HTTP/2) section 10.5.1

From: 安福一樹 <[email-masked]>
Date: Fri, 06 Apr 2018 06:57:40 +0000

Hello

I am a developer of this HTML5 game upload site https://game.nicovideo.jp/atsumaru/ <https://game.nicovideo.jp/atsumaru/>
I have some question about RFC7540 section 10.5.1.
Why server "CAN" send an HTTP 431 status code when receives a larger header block (not MUST)?

In http/1.1 connection, server MUST respond 4xx status code when receives a larger header.(RFC7230)
So, if user access a site that can upload any javascript code, and get large cookies, then we can send customized HTTP 4xx response which contains erasing cookie code.
But, in http/2, server does not need to send HTTP 431 response, we will not have a chance to erase cookies.

In actual implementaion, nginx will terminate http/2 session with ENHANCE_YOUR_CALM error without any HTTP responses, so chrome will display "cannot connect to server", 
So, we cannot send response which contains erasing cookie code to user who plays a game contains "Cookie Bomb".

So, we have two questions.
first question: why changed the text from "CAN" to "MUST" when recieves a large cookies(headers).
second question: is this problem an implementation issue or a specification problem?

Sincerely,

Kazuki Yasufuku

-- 
*******************************************

Kazuki Yasufuku

Software Engeneer, UGC game platform section

DWANGO Co., Ltd.
E-MAIL:[email-masked] <mailto:[email-masked]>
*******************************************

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 6 Apr 2018 15:29:05 +0000

Hi,

IMO from reading the specs, both RFC 7540 and 7230 apply. 7230 talks about individual header fields (and sets), while 7540 talks about the header block (the serialized combination of all header fields for a particular message).

FWIW it sounds like an implementation/config issue to me. What is the size of SETTINGS_MAX_HEADER_LIST_SIZE for these connections? It seems like that size is smaller than the check in your HTTP/1.1 case.

I’m inclined to think that it is valid for a server to first send a 4XX response, followed by a GOAWAY with ENHANCE_YOUR_CALM. I’d be interested to hear what others think.

Regards
Lucas

From: 安福一樹 [mailto:[email-masked]]
Sent: 06 April 2018 07:58
To: [email-masked]
Subject: Question about RFC7540 (HTTP/2) section 10.5.1

Hello

I am a developer of this HTML5 game upload site https://game.nicovideo.jp/atsumaru/
I have some question about RFC7540 section 10.5.1.
Why server "CAN" send an HTTP 431 status code when receives a larger header block (not MUST)?

Sincerely,

Kazuki Yasufuku

DWANGO Co., Ltd.
E-MAIL:[email-masked]<mailto:[email-masked]>
*******************************************

----------------------------

http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

---------------------

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 6 Apr 2018 21:34:24 +0200

Hi Lucas,

On Fri, Apr 06, 2018 at 03:29:05PM +0000, Lucas Pardue wrote:
> FWIW it sounds like an implementation/config issue to me. What is the size of
> SETTINGS_MAX_HEADER_LIST_SIZE for these connections? It seems like that size
> is smaller than the check in your HTTP/1.1 case.

Very possible. It's also possible that a conversion from H2 to H1 fails on
the H1 side and that this failure is not easily reported to the client. By
the way, the spec says the value in this setting is only advisory.

> I'm inclined to think that it is valid for a server to first send a 4XX
> response, followed by a GOAWAY with ENHANCE_YOUR_CALM. I'd be interested to
> hear what others think.

Yep, that would be the best whenever possible.

Kazuki, regarding the reason why it's a CAN and not a MUST, it's very likely
because 431 arrived way later than most deployed servers, and that it's
always possible to fall back to 400 for any 4xx . Even if the spec would
change, that would not suddenly upgrade deployed servers. Your implementation
needs to protect itself against any misbehaving agent. Here a GOAWAY may be
the only solution this agent has in this particular situation due to a
technical constraint. Maybe if you know that your application sometimes bugs
and bombs with cookies, using various request rejection signals (such as 4xx,
RST_STREAM and GOAWAY) could help detect the problem and work around it.

Regards
Willy

---

From: Kazuho Oku <[email-masked]>
Date: Sat, 7 Apr 2018 14:34:41 +0900

2018-04-07 0:29 GMT+09:00 Lucas Pardue <[email-masked]>:
> Hi,
>
>
>
> IMO from reading the specs, both RFC 7540 and 7230 apply. 7230 talks about
> individual header fields (and sets), while 7540 talks about the header block
> (the serialized combination of all header fields for a particular message).

I share the same view.

A H2 server can buffer the HEADERS frame and the following
CONTINUATIONS frames until it sees the END_HEADERS flag, and then try
to decode the frames at once. If the headers block is too large, the
server immediately sends a GOAWAY with ENHANCE_YOUR_CALM without
decoding HPACK.

We cannot have that kind of implementation if the specification
mandated the use of 431, because you cannot tell if the size of the
header fields were actually exceeding the limit without decoding
HPACK.

>
>
>
> FWIW it sounds like an implementation/config issue to me. What is the size
> of SETTINGS_MAX_HEADER_LIST_SIZE for these connections? It seems like that
> size is smaller than the check in your HTTP/1.1 case.
>
>
>
> I’m inclined to think that it is valid for a server to first send a 4XX
> response, followed by a GOAWAY with ENHANCE_YOUR_CALM. I’d be interested to
> hear what others think.

I would argue that there's no merit in sending a H2 level error (i.e.
ENHANCE_YOUR_CALM) in case you send a HTTP-level error (i.e. 4xx).

> So, if user access a site that can upload any javascript code, and get large
> cookies, then we can send customized HTTP 4xx response which contains
> erasing cookie code.
>
> In actual implementaion, nginx will terminate http/2 session with
> ENHANCE_YOUR_CALM error without any HTTP responses, so chrome will display
> "cannot connect to server",
>
> E-MAIL:[email-masked]
>
> This e-mail (and any attachments) is confidential and may contain personal
> views which are not the views of the BBC unless specifically stated.

-- 
Kazuho Oku

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 7 Apr 2018 09:08:44 +0000

Hi Kazuho,

> ENHANCE_YOUR_CALM) in case you send a HTTP-level error (i.e. 4xx).

That's a good point and I kind of agree. It's just that practically, some of the UA scaffolding for handling Application-layer errors (sometimes referred to as network errors) are not in place or mature. I was reminded recently of the W3C work on Network Error Logging [1] that maybe helps in such cases?

My other thought process was that a single client error may not be cause for a connection error, rather a pattern of behaviour could cause the connection to be closed. However, on reflection in this case from the purest perspective I agree.

Lucas

[1] https://wicg.github.io/network-error-logging/

-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 7 Apr 2018 21:59:04 +1200

> My other thought process was that a single client error may not be cause for a connection error, rather a pattern of behaviour could cause the connection to be closed. However, on reflection in thi…

Not just from a purist perspective. If any one header block is too large
to process properly, or was not processed by the recipient, it screws up
the HPACK state - causing corruption of all following HEADERS and
CONTINUATION frames. So yes it MUST be the last thing on that
connection. The only optional part is whether the 4xx status code is
produced before termination.

AYJ

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 7 Apr 2018 10:46:15 +0000

Hi Amos,

> produced before termination.

I think we are agreeing, thanks for the highlighting that point.

To go back to the specific problem with this in mind. It seems that the H2 layer is improperly configured to support the HTTP semantic of this particular use case. Which leads to the connection error; the implementation choosing not to execute the optional 4xx status doen't meet the client expectation.

Regards
Lucas