Subject: draft-ietf-httpbis-header-structure-00, unicode range

From: Kari Hurtta <[email-masked]>
Date: Tue, 13 Dec 2016 19:33:26 +0200 (EET)

2.  Definition of HTTP Header Common Structure
https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-2

|     unicode_string = * unicode_codepoint
|             # XXX: Is there a place to import this from ?
|             # Unrestricted unicode, because there is no sane
|             # way to restrict or otherwise make unicode "safe".

What is range of unicode_codepoint ?

Next section implies that it is only plane 0 ?
( range 0x0000 - 0xFFFF )

3.  HTTP/1 Serialization of HTTP Header Common Structure
https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-3

|     h1_unicode_string = DQUOTE *(
|                         ( "\" DQUOTE )
|                         ( "\" "\" ) /
|                         ( "\" "u" 4*HEXDIG ) /
|                         0x20-21 /
|                         0x23-5B /
|                         0x5D-7E /
|                         UTF8-2 /
|                         UTF8-3 /
|                         UTF8-4
|                         ) DQUOTE
|     # This is UTF8 with HTTP1 unfriendly codepoints
|     # (00-1f, 7f) neutered with \uXXXX escapes.

here seems unicode to limited to plane 0
( 0x0000 - 0xFFFF )

Or is unicode values > 0xFFFF
encoded with surrogates  (values 0xd8000 - 0xdffff) ?
( UCS-2 or UTF-16 is used )

/ Kari Hurtta

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 13 Dec 2016 19:54:19 +0200

> |     h1_unicode_string = DQUOTE *(

Well, that production lists UTF8-4, which is presumably 4-byte UTF-8
sequences, and all valid ones are astral plane codepoints.

Also, if one is to use native unicode (UTF-8), escape sequences are
only needed for range 0-159. Or otherwise you also need escapes for
astral planes (and I hope the escape system there would be more sane
than the one JSON has...)

-Ilari

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 13 Dec 2016 20:08:59 +0200 (EET)

Ilari Liusvaara <[email-masked]>: (Tue Dec 13 19:54:19 2016)
> sequences, and all valid ones are astral plane codepoints.

OK.
 
> only needed for range 0-159. 

    Then that 
	 ( "\" "u" 4*HEXDIG ) /
    is overkill.

>                               Or otherwise you also need escapes for

That ( "\" "u" 4*HEXDIG ) produces same than JSON have 
when UTF-16 is used:

https://tools.ietf.org/html/rfc4627#section-2.5

   To escape an extended character that is not in the Basic Multilingual
   Plane, the character is represented as a twelve-character sequence,
   encoding the UTF-16 surrogate pair.  So, for example, a string
   containing only the G clef character (U+1D11E) may be represented as
   "\uD834\uDD1E".

/ Kari Hurtta

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 13 Dec 2016 21:28:47 +0000

--------
In message <[email-masked]>, Kari Hurtta wri

>What is range of unicode_codepoint ?

As far as I know, UNICODE does not have a firm upper end, but
everybody _expects_ 32 bits to be enough for everybody.

Since section two is the abstract datamodel, that's the best we can
do there.

>[...]

That was the plan.

Not a particular good plan, as evindenced by the fact that I forgot
to write that, and that JSON has seen interop issues with parsers
missing that detail.

I will add text about it.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 13 Dec 2016 21:43:15 +0000

--------
In message <[email-masked]>, Ilari Li

>sequences, and all valid ones are astral plane codepoints.

My impression was that UTF8 and 8-bit clean HTTP/1 got shot down
in previous discussions, but I left UTF8 here for now, pending a
more structured decision making on this.

I see us having four options, in my order of preference:

1) Forbid Unicode in headers.

2) Take UTF8 out and leave all (non-ASCII) unicode to the \uxxxx
   escape mechanism.

3) Leave UTF8 in, and make it clear that it may or may not work, so
   that people can use it in controlled environments.

4) Leave UTF8 in, and specify how to indicate/negotiate if it can be used.

>than the one JSON has...)

Any suggestions ?

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 13 Dec 2016 23:42:23 +0200

> everybody _expects_ 32 bits to be enough for everybody.

Actually, it does: 10FFFD is the last codepoint in Unicode (it is
actually allocated as part of PUA).

IIRC, Unicode has exactly 1,111,998 codepoints in total (most of those
are unallocated). 
 
> to write that, and that JSON has seen interop issues with parsers

Also, note that the surrogate mechanism can only encode up to plane 16
(that's the reason why unicode only has 17 planes!)

And I suppose that the surrogates MUST be paired properly (JSON actually
does not require this).

-Ilari

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 13 Dec 2016 22:06:29 +0000

--------
In message <[email-masked]>, Ilari Li

>IIRC, Unicode has exactly 1,111,998 codepoints in total (most of those

Future archaeologists will stare at that number and conclude that
"It must have had religious significance for them" :-)

>And I suppose that the surrogates MUST be paired properly (JSON actually

Yes.  I'm firmly in the "Jon Postel was wrong" camp of protocol design
these days.

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 14 Dec 2016 08:28:24 +0200 (EET)

Poul-Henning Kamp <[email-masked]>: (Tue Dec 13 23:43:15 2016)
  
> >than the one JSON has...)

I think that one escape sequence is more sane than something like
\uD834\uDD1E  for one unicode codepoint.

> Any suggestions ?

Ilari Liusvaara told that 10FFFD is the last codepoint. So 6
hex digits is sufficient.

Either
	( "\" "X" 6*HEXDIG )

or

	 ( "\" "X" 1*6HEXDIG "#" )
or

	 ( "\" "#" 1*6HEXDIG "#" )

or   escape characters may be some other also. 
     This was my first suggestion. 

I did not suggested \u  or \U  because these
two are used with different length.

/ Kari Hurtta

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 07:43:49 +0100

> Yes.  I'm firmly in the "Jon Postel was wrong" camp of protocol design

Actually, we shouldn't use surrogate pairs *at all*.

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 07:42:08 +0000

--------
In message <[email-masked]>, Kari Hurtta writes

>hex digits is sufficient.

I'm totally agnostic on this one, but would lean on doing it
like JSON according to Occams Razor.

If we do something different, does the HPACK-Huffman efficiency matter ?

>	( "\" "X" 6*HEXDIG )

HPACK: 19 + 8 + 6 * 5.625-ish = 61-ish bits
(lowercase 'x' would save a bit)

>	 ( "\" "X" 1*6HEXDIG "#" )

HPACK: 19 + 8 + 3-ish * 5.625-ish + 8 = 51-ish bits
(lowercase 'x' would save a bit)

>	 ( "\" "#" 1*6HEXDIG "#" )

HPACK: 19 + 12 + 3-ish * 5.625-ish + 12 = 60-ish bits

	( "\" "u" 4*HEXDIG )

HPACK: 19 + 6 + 4 * 5.625-ish = 47-ish bits

---

From: Martin J. Dürst <[email-masked]>
Date: Wed, 14 Dec 2016 16:45:37 +0900

> IIRC, Unicode has exactly 1,111,998 codepoints in total (most of those

Yes. And Unicode never had more (at some time, it had less). ISO-10646 
at one time had a 32-bit architecture, but that was fixed quite a while 
(~10 years?) ago. It is now also the same as Unicode.

Regards,   Martin.

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 14 Dec 2016 18:46:04 +1000

> like JSON according to Occams Razor.
​There's a recursive argument to be made here about accepting one of JSON's
flaws, so why not another? (In my experience surrogate pairs are never not
a problem.)​

> HPACK: 19 + 6 + 4 * 5.625-ish = 47-ish bits
​Unless we're using a different ABNF, doesn't "X" match both %x58 and %x78
?  I'm sure this doesn't intend to be case-insensitive.

If efficiency matters we're probably better off using a sentinel character
like "%" that encodes down much smaller. (Maybe we should be more like
%DOS% here... ;)

If we're looking for inspiration elsewhere, why not C99?

      "\" %x75 1*4HEXDIG
    / "\" %x55 1*6HEXDIG  ; C99 accepts 1*8

Even removing the '1*' from the recurrence rules still gives ~47 bits for
most uses and ~60 for astral plane emoji. (That's better than \#XXXXXX# for
large codepoints, by the way)

Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 09:21:28 +0000

--------
In message <CACweHNDKgWQewZHb=Kz3_2=[email-masked]>

>    / "\" %x55 1*6HEXDIG  ; C99 accepts 1*8

The variable length means these run into the mess of "what if the
next character is a hex digit?".

That's a much bigger issue when we're talking about code spitting
out strings than when it is programmers typing them in.

I would prefer to make them fixed length:

	"\" "u" 4HEXDIG
	"\" "U" 6HEXDIG

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Dec 2016 23:28:59 -1000

> next character is a hex digit?".

I assume that if the next character is a hex digit, you have to
exhaust the sequence by left padding with 0.  It makes the grammar a
little harder to parse though.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 14 Dec 2016 09:59:52 +0000

> I would prefer to make them fixed length:
IETF has published BCP 137, which should be followed, unless there is a 
very good reason not to:
  https://www.rfc-editor.org/bcp/bcp137.txt

See section 5.1.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 11:15:28 +0100

>> than the one JSON has...)

3) seems like the right choice to me.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 11:14:41 +0100

> little harder to parse though.

...and very easy to get wrong.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 11:21:35 +0100

On 2016-12-14 10:59, Alexey Melnikov wrote:
>> In message
>> <CACweHNDKgWQewZHb=Kz3_2=[email-masked]>
>> , Matthew Kerwin writes:
>>

Which says:

>    EmbeddedUnicodeChar =  %x5C.75.27 4*6HEXDIG %x27
>       ; starting with lowercase "\u" and "'" and ending with "'".
>       ; Note that the encodings are considered to be abstractions
>       ; for the relevant characters, not designations of specific
>       ; octets.
>
>    HEXDIG =  "0" / "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9" /
>       "A" / "B" / "C" / "D" / "E" / "F"
>       ; effectively identical with definition in RFC 5234.

Has this ever been used in a protocol?

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 10:23:51 +0000

--------
In message <[email-masked]>, Alexey Melnikov wr

>  https://www.rfc-editor.org/bcp/bcp137.txt

So many RFCs, so little time...

That works for me, but HPACK is not a big fan of it:

	19 + 6 + 11 + [4:6] * [5:6] + 11 = [67:83]

Is that a show-stopper for anybody in Asia ?

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 11:35:16 +0100

> That works for me, but HPACK is not a big fan of it:

Why are we concerned with HPACK? Wouldn't we convert to raw UTF-8 before 
HPACKing?

Best regards, Julian

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 14 Dec 2016 10:38:28 +0000

> Has this ever been used in a protocol?
Some:
https://datatracker.ietf.org/doc/rfc5137/referencedby/

This was also extensively used in other RFCs without referencing the BCP.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 11:46:32 +0100

On 2016-12-14 11:38, Alexey Melnikov wrote:
> ...

Actually, one.

> This was also extensively used in other RFCs without referencing the BCP.

Example?

The reason why I'm asking is because the notation

  \u'HHHH' or \u'HHHHHH'

strikes me as:

1) verbose

2) potentially problematic because of the use of the single quote (which 
might require extra escaping in some contexts)

Best regards, Julian

PS: and, as a nit, it's strange that the syntax uses delimiters but 
doesn't allow sequences of 1 to 3 HEXDIGs...

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 10:51:20 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>Why are we concerned with HPACK? Wouldn't we convert to raw UTF-8 before 

Well, UTF-8 would also go through HPACK, but by eye-ball it seems
that it would be more efficient.

But it would kill the "HPACK doesn't molest tunneled H1 headers" idea ?

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Dec 2016 22:37:45 +1100

> that it would be more efficient.

If you have lots of ASCII still, you can probably Huffman encode,
though if you have lots of non-ASCII, you need to watch out: a three
octet UTF-8 encoded codepoint turns into (worst case) 82 bits.  Best
case is 58 bits (both of which are invalid, so maybe not).

I can't remember, is there actually a good reason why we can't just
start shoving UTF-8 in header fields?  I mean, h2 is probably OK with
this.

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 14 Dec 2016 21:53:45 +1000

> might require extra escaping in some contexts)
​Yes.

It says that "forms that use explicit string delimiters are generally
preferred over other alternatives. In many contexts, symmetric paired
delimiters are easier to recognize and understand than visually unrelated
ones." So brackets are good.

And while it advises against using Perl's \x{NNNN...} syntax (because of
potential ambiguities with two-digit hex codes), it doesn't say anything at
all about \u{N...}

Curly braces cost 14+15 bits in HPACK, parentheses 10+10 (incidentally
cheaper than single quotes, which are 11+11). It's also convenient that
little 'u' is one bit cheaper than little 'x'.

I don't think parentheses are at too much risk of needing escaping, so it
seems like the solution that goes with BCP 137, and compresses alright with
HPACK, is:

    %x5c.75.28 1*6HEXDIGIT %x29

It's still a little bit clunky for things like "Stra\u(df)e", but not so
bad for emoji "\u(1f602)" and somewhere in between for Hiragana "
\u(3053)\u(3093)\u(306b)\u(3064)".

Cheers​

> doesn't allow sequences of 1 to 3 HEXDIGs...
​Having just written "\u(df)" I kind of understand; it really feels like
I'm describing an octet rather than a codepoint. I don't think there's a
*technical* reason, though.  Is it alright to see "\u(9)" or an equivalent
in text?
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2016 12:53:27 +0100

> start shoving UTF-8 in header fields?  I mean, h2 is probably OK with

Some APIs assume ISO-8859-1, so unexpected things might happen (of 
course that's independent of the actual transport).

Best regards, Julian

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 14 Dec 2016 22:04:03 +1000

On 14 December 2016 at 21:53, Julian Reschke <[email-masked]> wrote:

> Some APIs assume ISO-8859-1, so unexpected things might happen (of course
> that's independent of the actual transport).
>
> Best regards, Julian
>
>
​Particularly since HTML4 taught us that "ISO-8859-1" means "Windows-1252",
which actually has values for most of \x80-\x9F

Cheers

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 12:56:37 +0000

--------
In message <CACweHNBYf-UuxsKNxYakt22rgku9xEP4YK4yL2R+=[email-masked]>

>    %x5c.75.28 1*6HEXDIGIT %x29

That is so obvious that I think we need to ask if BCP137 considered
it and rejected it ?

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Dec 2016 12:54:59 +0000

--------
In message <CABkgnnWzOhkznH2HzweNegYo4dDHE+DT0PM=[email-masked]>

>start shoving UTF-8 in header fields?  I mean, h2 is probably OK with

You mean "h2 end to end" ?  Yes, probably.

But what about H2->H1 and H1->H2 proxies/load-balancers/etc ?

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 14 Dec 2016 19:39:58 +0200 (EET)

Matthew Kerwin <[email-masked]>: (Wed Dec 14 13:53:45 2016)
> \u(3053)\u(3093)\u(306b)\u(3064)".

I think that this is best suggestion so far.

But can this also be shorter ?

     %x5c.28 1*6HEXDIGIT %x29

Makes

	\(3064)

{ Yes, it is not visible that this is hexadecimal. }

Although

works for me.
 
> *technical* reason, though.  

Yes.

>                              Is it alright to see "\u(9)" or an equivalent
> in text?

	Or is that "\(9)" alright if 'u' is also dropped.

If that wanted to be avoid, that means

	%x5c.75.28 3*6HEXDIGIT %x29

or

	%x5c.28 3*6HEXDIGIT %x29

on my newest suggestion.

/ Kari Hurtta

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 15 Dec 2016 11:57:40 +1000

> > potential ambiguities with two-digit hex codes), it doesn't say anything

​I have should noted here that Ruby uses this \u{N...} syntax, including
the lower limit of one hexadecimal digit.  This is a valid string literal
in Ruby:

"\u{df}\u{9}\u{1f602}"​

> { Yes, it is not visible that this is hexadecimal. }
​There is precedent, although I'm not sure if it's a good precedent: the
"content" attribute in CSS uses:

    %5c 1*6HEXDIGIT

...which is both undelimited (which I oppose) and without an explicit
hexadecimal indicator (about which I'm mostly ambivalent.)​

>         EmbeddedUnicodeChar =  %x5C.75.27 4*6HEXDIG %x27
​I suppose it comes down to a question of which data we want to target for
optimisation, and then taking measurements and evaluating them.

It sounds like Julian thinks «%x5c.75 DELIM 1*6HEXDIGIT DELIM» "\u(abc)" is
verbose, and we don't have many opinions yet on «%x5c.28 1*6HEXDIGIT %x29​»
"\(abc)"

I'm not sure at what point this decision becomes so minor that it's just
paint on a bike shed. :)

>         %x5c.75.28 3*6HEXDIGIT %x29
​Left-padding a with zeroes to make three digits screams "octal" at me,
even when they're not all octal digits, which elicits an even stronger
Pavlovian response. I think it has to be either
1*6
or
4*6
, and I lean towards
1*6
.

Cheers

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 16 Dec 2016 07:14:13 +0900

> \uD834\uDD1E  for one unicode codepoint.

Agreed.

Surrogate pairs are a remnant of old systems that expected all unicode
characters would fit into 16-bit, which turned out to be false. They
make sense only when the underlying system is using UTF-16.

Considering the fact that most of the programming languages (and
operating systems) are moving away from UTF-16, I think creating a new
standard that permits the use of surrogate pairs would be considered a
bad move in the long run.

-- 
Kazuho Oku

---

From: Daurnimator <[email-masked]>
Date: Wed, 21 Dec 2016 17:26:32 +1100

On 15 December 2016 at 12:57, Matthew Kerwin <[email-masked]> wrote:
> I have should noted here that Ruby uses this \u{N...} syntax, including the
> lower limit of one hexadecimal digit.  This is a valid string literal in
> Ruby:
>
> "\u{df}\u{9}\u{1f602}"

Lua also has this style of unicode codepoint escaping using curly braces.

>From lua reference manual:
> The UTF-8 encoding of a Unicode character can be inserted in a literal string with the escape sequence \u{XXX} (note the mandatory enclosing brackets), where XXX is a sequence of one or more hexadecimal digits representing the character code point.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Dec 2016 09:37:26 -0500

> On 14 Dec. 2016, at 7:54 am, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

Furthermore (and it bears repeating), the ends are never just "HTTP." 

On the server side, they're a mash of CGI, FastCGI and various other interfaces to languages like Perl, Python, PHP and Ruby, each with their own galaxy of library modules, frameworks and such.

On the client side, things get easier because of the relative alignment between browsers*, but you still have to consider non-browser clients, including spiders, robots, scrapers -- and the various libraries and infrastructure they use.

Both sides are implemented by intermediaries, whether that be "forward" proxies, "reverse" ones, CDNs, load balancers, firewalls, or on-machine virus scanners (ew). If you're really lucky, they might pass through an ICAP hop or two.

Off-path, you need to consider logging and monitoring software, as well as configuration interfaces that allow headers to be manipulated (e.g. through Web form -- that should be fun).

Potentially, all of these interfaces and pieces of software touch HTTP headers, and might assume that they are ASCII, 8859-1, UTF-8, or a bytestring. 

That's not to say that we can't use more than the least common denominator (ASCII), but we don't know how much trouble doing so will cause. And, as discussed previously, there aren't a lot of use cases for non-ASCII header values in standards (because few have a payload that's exposed to end users), so the reward for taking that risk is questionable.

Cheers,

* If you believe in Fetch.

--
Mark Nottingham   https://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 23 Dec 2016 16:13:29 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>And, as discussed previously, there aren't a lot of use cases for 
>non-ASCII header values in standards (because few have a payload that's 
>exposed to end users), so the reward for taking that risk is 
>questionable.

But isn't that an almost circular[1] argument, when there is no
safe standardized way to put non-ASCII into header values in the
first place ?

Anyway, here is my current thinking:

a) Remove the UTF8 option
-------------------------

Having UTF8 as an option for the H1 serialization is unlikely to
significantly increase the number of paths on which you can use it.

The subset where it will work, will be the same subset with or without
our blessing:  End-to-End fully confined and controlled paths.

b) Add BCP137's recommendation unchanged (%x5C.75.27 4*6HEXDIG %x27)
--------------------------------------------------------------------

A number of alternatives have been proposed, but I do not think the
relatively minor increase in HPACK and H1 afficiency they offer 
warrants "needlessly multiplying entities"[2].

Absent really convincing arguments, that will be my next edit.

Poul-Henning

[1] "Almost circular" is a lot more complex than most people imagine.
Highly recommended reading: http://press.princeton.edu/titles/8624.html

[2] Occams Razor.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Dec 2016 11:36:59 -0500

> On 23 Dec. 2016, at 11:13 am, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

There is -- RFC5987 encoding. Not pretty or efficient, but implemented and interoperable. Used in Content-Disposition, Link (although not much), and not much else AFAIK (Julian?).

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 Dec 2016 18:19:18 +0100

> There is -- RFC5987 encoding. Not pretty or efficient, but implemented and interoperable. Used in Content-Disposition, Link (although not much), and not much else AFAIK (Julian?).

It's used in the new Digest spec, but I don't believe it's implemented yet.

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 24 Dec 2016 09:04:44 +0000

--------
In message <[email-masked]>, Julian Reschke writes
:

>> There is -- RFC5987 encoding. Not pretty or efficient, but
>> implemented and interoperable. Used in Content-Disposition, Link
>> (although not much), and not much else AFAIK (Julian?).
>

(RFC5987 uses %XX and UTF-8)

BCP137 doesn't really say anything about that, except a mild scorn for
UTF8 in general in preference for 'human readable' unicode points, and
I unless we want to incorporate RFC5987 into header-structure, we should
probably lean more on BCP137.

I asked previously if header-structure should incorporate RFC5987,
but I understodd the answer as "don't bother" ?

(The main problem in absorbing RFC5987 is that a trailing '*' on a
dictionary key gets a special meaning, and '*' is a valid character
in a RFC7230::token on which I based header-structure's identifier.)

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 25 Dec 2016 08:59:58 -0500

> On 13 Dec 2016, at 4:43 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

When you say "take out / leave in" above, that's only in the context of the HTTP/1 serialisation, correct?

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 25 Dec 2016 14:31:28 +0000

--------
In message <[email-masked]>, Mark Nottingham wri

>the HTTP/1 serialisation, correct?

Yes, sorry if that wasn't clear.