Subject: Translating CR LF to HTTP/1 headers

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 4 Sep 2013 00:31:27 +0900

In http://http2.github.io/http2-spec/#rfc.section.10.3

"""
An intermediary that performs translation into HTTP/1.1 cannot
alter the semantics of requests or responses. In particular,
header field names or values that contain characters not
permitted by HTTP/1.1, including carriage return (U+10) or line
feed (U+13) MUST NOT be translated verbatim.
"""

So if U+10 and/or U+13 are included in HTTP/2 headers, what
should intermediary do to translate them into HTTP/1?  Just
remove them or replace them with safe characters? The meaning of
the headers may change with these changes, especially if those
characters are permitted in HTTP/2 headers.  Returning 5xx
response may be an option. But there is very interesting
situation. If all machines in the path are HTTP/2, then the
request succeeds, but if one of them is HTTP/1 then request
fails.

Best regards,

Tatsuhiro Tsujikawa

---

From: Julian Reschke <[email-masked]>
Date: Tue, 03 Sep 2013 17:47:10 +0200

> request succeeds, but if one of them is HTTP/1 then request

CR and LF *are* allowed in HTTP/1.1 field *values* (not names), as per 
the OBS-FOLD ABNF production. 
(<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-23.html#header.fields>)

Best regards, Julian

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 4 Sep 2013 00:53:02 +0900

On Wed, Sep 4, 2013 at 12:47 AM, Julian Reschke <[email-masked]>wrote:

>> In http://http2.github.io/http2-**spec/#rfc.section.10.3<http://http2.github.io/http2-spec/#rfc.section.10.3>
>>
>> """
> CR and LF *are* allowed in HTTP/1.1 field *values* (not names), as per the
> OBS-FOLD ABNF production. (<http://greenbytes.de/tech/**
> webdav/draft-ietf-httpbis-p1-**messaging-23.html#header.**fields<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-23.html#header.fields>
> >)
>
>
Does it mean that intermediary should translate CR and LF in the HTTP/2
field-value to OBS-FOLD sequence?

Best regards,

Tatsuhiro Tsujikawa

---

From: Julian Reschke <[email-masked]>
Date: Tue, 03 Sep 2013 18:45:57 +0200

On 2013-09-03 17:53, Tatsuhiro Tsujikawa wrote:
>
>
>
> On Wed, Sep 4, 2013 at 12:47 AM, Julian Reschke <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>         In http://http2.github.io/http2-__spec/#rfc.section.10.3
>         <http://http2.github.io/http2-spec/#rfc.section.10.3>
>
>         """
>     CR and LF *are* allowed in HTTP/1.1 field *values* (not names), as
>     per the OBS-FOLD ABNF production.
>     (<http://greenbytes.de/tech/__webdav/draft-ietf-httpbis-p1-__messaging-23.html#header.__fields
>     <http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-23.html#header.fields>>)
>
>

Actually, to SP. See 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-23.html#rfc.section.3.2.4.p.6>:

"A proxy or gateway that receives an obs-fold in a response message that 
is not within a message/http container MUST either discard the message 
and replace it with a 502 (Bad Gateway) response, preferably with a 
representation explaining that unacceptable line folding was received, 
or replace each received obs-fold with one or more SP octets prior to 
interpreting the field value or forwarding the message downstream.

A user agent that receives an obs-fold in a response message that is not 
within a message/http container MUST replace each received obs-fold with 
one or more SP octets prior to interpreting the field value."

Best regards, Julian

---

From: Martin Thomson <[email-masked]>
Date: Tue, 3 Sep 2013 11:02:05 -0700

On 3 September 2013 08:47, Julian Reschke <[email-masked]> wrote:
> OBS-FOLD ABNF production.

Let's be really clear.  What do you consider in this case to be the
"value"?  The lexical form or the value-space form?  Because if I sent
you something like:

Header: value1,
    value2

The lexical value is one thing, but there is no obligation on a stack
or intermediary to preserve obs-fold.  The value-space form is simply
"value1, value2" with a single space replacing all that extra fluff.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 03 Sep 2013 20:04:11 +0200

> "value1, value2" with a single space replacing all that extra fluff.

Si.

Best regards, Julian

---

From: Martin Thomson <[email-masked]>
Date: Tue, 3 Sep 2013 11:24:56 -0700

> request succeeds, but if one of them is HTTP/1 then request

This is an important point, and I'll have to lean on Julian's
expertise for this one, since he knows the header encoding mess better
than I.  A value can include U+13 and U+10, but only if they are
escaped using the encoding described in RFC5987.  Any U+10 or U+13
that appears in the value of a header field is therefore part of
obs-fold, and can be replaced by SP.  We're saying here that it MUST
be replaced by SP.

I don't think that any grammar permits CR/LF as anything other than as
part of obs-fold.  e.g.,
http://tools.ietf.org/html/rfc2616#section-2.2, which specifically
says:

   [...] A CRLF is allowed in the definition of TEXT only as part of a header
   field continuation.

httpbis is much stricter about obs-fold:
tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-23#section-3.2.4

---

From: James M Snell <[email-masked]>
Date: Tue, 3 Sep 2013 14:10:45 -0700

It's very important to note that the grammar rules for existing
headers have not yet been changed for 2.0. The typed codecs introduce
possible changes that could deal with this fairly easily, but so long
as we're using the existing definitions, U+10/U+13 are handled exactly
as Julian describes. If they are encoded in some way (i.e. RFC5987)
then they are handled as just part of the value, if they are encoded
specifically as U+10/U+13 then they're obs-fold.

- James

On Tue, Sep 3, 2013 at 11:24 AM, Martin Thomson
<[email-masked]> wrote:
>[snip]

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 4 Sep 2013 09:40:09 +1000

“…carriage return (U+10) or line feed (U+13)…”
 [http://http2.github.io/http2-spec/#rfc.section.10.3]

The numbers 10 & 13 are the wrong way round, and "U+" is always a prefix for hex digits, never decimal. Can we change this phrase to:

“…carriage return (U+000D) or line feed (U+000A)…”

--
James Manger

---

From: Martin Thomson <[email-masked]>
Date: Tue, 3 Sep 2013 16:45:26 -0700

> “…carriage return (U+000D) or line feed (U+000A)…”

Yeah, my bad.  Fixed.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Thu, 5 Sep 2013 01:20:49 +0900

Thank you for the clarification. How about clearly state that U+000D and
U+000A in header field must be replaced with U+0020 in that section? It
will spare the time for the readers to go around and check obs-fold
production.
Just like this:

"""
...
In particular, header field names or values that contain characters not
permitted by HTTP/1.1, including carriage return (U+000D) or line feed
(U+000A) MUST NOT be translated verbatim.
The carriage return (U+000D) and line feed (U+000A) in header field values
are considered instances of the obs-fold production and MUST be replaced
with space (U+0020).
"""

Best regards,

Tatsuhiro Tsujikawa

---

From: Martin Thomson <[email-masked]>
Date: Wed, 4 Sep 2013 09:39:37 -0700

On 4 September 2013 09:20, Tatsuhiro Tsujikawa <[email-masked]> wrote:
> U+000A in header field must be replaced with U+0020 in that section? It will
> spare the time for the readers to go around and check obs-fold production.

httpbis already has great text on this.  This is really just a pointer
to the decision in
http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-23#section-3.2.4

A reference to that makes sense, see b5dda02.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Thu, 5 Sep 2013 01:45:42 +0900

On Thu, Sep 5, 2013 at 1:39 AM, Martin Thomson <[email-masked]>wrote:

> will

It works for me. Thanks.

Best regards,

Tatsuhiro Tsujikawa