Subject: Re: Dealing with Invalid UTF-8

From: Julian Reschke <[email-masked]>
Date: Mon, 02 Sep 2013 16:40:38 +0200

Hi,

I note that compression spec still pretends that field values are 
encoded in UTF-8.

Have we agreed on this? If so, how do you forward existing header fields 
that may not be UTF-8???

Best regards, Julian

On 2013-08-16 18:31, Roberto Peon wrote:
> Yup.
>
>
> On Fri, Aug 16, 2013 at 2:30 AM, Julian Reschke <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     On 2013-08-14 00:12, James M Snell wrote:
>
>         https://github.com/http2/__http2-spec/issues/232
>         <https://github.com/http2/http2-spec/issues/232>
>
>         The current header compression draft states that header field values
>         are UTF-8. However, the spec says nothing about how to deal with
>         overlong encodings, invalid UTF-8 octet sequences or valid UTF-8
>         sequences that encode invalid Unicode codepoints.
>
>         I recommend stating that any of these conditions ought to result
>         in an
>         error. An encoder MUST NOT output any of these; and a decoder
>         ought to
>         signal a connection error if encountered.
>
>
>     ...whether this is an issue or not depends on what we decide we
>     respect to whether field values are octet sequences or strings...
>
>     Best regards, Julian
>
>
>

---

From: James M Snell <[email-masked]>
Date: Mon, 2 Sep 2013 07:48:52 -0700

The solution I'd prefer is to go with the typed codecs I've proposed, which
can be separated out from the main header compression algorithm and have a
clearly defined backwards compatibility story.
On Sep 2, 2013 7:40 AM, "Julian Reschke" <[email-masked]> wrote:

> Hi,
>
> I note that compression spec still pretends that field values are encoded
> in UTF-8.
>
>> <mailto:[email-masked]>**> wrote:
>>
>>         https://github.com/http2/__**http2-spec/issues/232<https://github.com/http2/__http2-spec/issues/232>
>>         <https://github.com/http2/**http2-spec/issues/232<https://github.com/http2/http2-spec/issues/232>
>> >
>>

---

From: Roberto Peon <[email-masked]>
Date: Mon, 2 Sep 2013 10:39:19 -0700

We have not agreed that the compressor should have to interpret
Utf-8, but we have not agreed that it shouldn't.

I think it should be doing byte-based  comparisons for values right now,
with the rest (e.g. other encodings) to be decided after the bare-bones is
working and interoperating.

-=R

in UTF-8.
that may not be UTF-8???

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 3 Sep 2013 22:11:03 +0900

On Tue, Sep 3, 2013 at 2:39 AM, Roberto Peon <[email-masked]> wrote:

> +1 for removing UTF-8 encoded values from the draft. Just interacting byte
array is a way to go. UTF-8 may bring some value if HTTP1 compatibility
issue was resolved, but not now.

Best regards,
Tatsuhiro Tsujikawa

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 3 Sep 2013 07:38:00 -0700

I also completely agree.  We may want to specify that some headers have
some particular encoding (UTF-8 or typed encoding, or something else) at
the HTTP layer, but the *compressor* should be defined to simply operate on
byte sequences.

On Tue, Sep 3, 2013 at 6:11 AM, Tatsuhiro Tsujikawa
<[email-masked]>wrote:

>> +1 for removing UTF-8 encoded values from the draft. Just interacting
> byte array is a way to go. UTF-8 may bring some value if HTTP1
> compatibility issue was resolved, but not now.
>
> Best regards,
> Tatsuhiro Tsujikawa
>
>
>
>> -=R
>>
>> > Have we agreed on this? If so, how do you forward existing header
>> fields that may not be UTF-8???
>> >
>> > Best regards, Julian
>> >
>> >
>> >