Subject: Re: Unknown text/* subtypes [i20]

From: Mark Nottingham <[email-masked]>
Date: Tue, 11 Mar 2008 16:50:45 +1100

My .02 -

Overall, I'm not intensely happy with this, but it does seem like the  
most practical way forward.

My biggest concern is that it places some fairly wide-reaching MUST- 
level requirements. Would downgrading them to SHOULD be workable? Can  
we refine their targets, e.g., instead of applying them to all  
recipients, target them at user-agents (and maybe origin servers too)  
as recipients?

We're also still needing the security considerations text WRT UTF-7,  
correct?

A few more editorial comments inline -

On 14/02/2008, at 2:39 PM, Roy T. Fielding wrote:
>
> 2.3.1.  Canonicalization and Text Media Types
>
>   Internet media types are registered with a canonical form and
>   defaults for the optional parameter values.  An ideal HTTP
>   entity-body would contain data formatted strictly according to that
>   canonical form.  However, HTTP does not require the sender to verify
>   that an entity-body is in canonical form prior to transfer.   
> Instead,
>   an HTTP recipient MUST be prepared to accept and properly interpret
>   several variances in the format of textual types, as described  
> below,
>   and treat other variances as errors.

This is a MUST, but the requirements about encoding below are MAYs,  
which is a bit odd...

>   The "charset" parameter (Section 2.1) is used with some media types
>   to indicate the character encoding of the data.  When a media type  
> is
>   registered with a default charset value of "US-ASCII", it MAY be  
> used
>   to label data transmitted via HTTP in the "iso-8859-1" charset (a
>   superset of US-ASCII) without including an explicit charset  
> parameter
>   on the media type.

This sentence doesn't read well; what is 'it'? 'label' is also not  
quite right, suggest 'indicate'. Also, who does the MAY apply to?

>  In addition, when a media type registered with a
>   default charset value of "US-ASCII" is received via HTTP without a
>   charset parameter or with a charset value of "iso-8859-1", the
>   recipient MAY inspect the data for indications of a different
>   character encoding and interpret the data accordingly if the  
> encoding
>   is a superset of US-ASCII or if the encoding can be determined  
> within
>   the first 16 octets of data and interpreted consistently thereafter.

This sentence is also very difficult. It may help to insert another  
MAY in between 'and' and 'interpret'.

>      Note: The first variance is due to a significant portion of early
>      HTTP user agents not parsing media type parameters and instead
>      relying on a then-common default encoding of iso-8859-1.  As a
>      result, early server implementations avoided the use of charset
>      parameters and user agents evolved to "sniff" for new character
>      encodings as the Web expanded beyond iso-8859-1 content.  The
>      second variance is due to a certain popular user agent that
>      employed an unsafe encoding detection and switching algorithm
>      within documents that might contain user-provided data (see
>      Section security.sniffing), the most common workaround for which
>      is to supply a specific charset parameter even when the actual
>      character encoding is unknown.
>
>   When in canonical form, media subtypes of the "text" type use CRLF  
> as
>   the text line break.  However, it is also commonplace for such types
>   to be transmitted in HTTP with CR or LF alone indicating a line
>   break and occasional for such types to be transmitted with a
>   character encoding that requires some other set of octet sequence(s)
>   to indicate a line break.  HTTP recipients MUST accept and properly
>   interpret CRLF, bare CR, and bare LF as indicating a line break when
>   encountered within an entity-body received via HTTP that is labeled
>   as a text type and provided in a character encoding that allows CRLF
>   to indicate a line break.
>
>      Note: Line breaks are specified in MIME with the expectation that
>      they are enforced during email message composition, when it is
>      scalable to ensure that every octet is placed in canonical form,
>      and with the anticipation that a message may be transmitted or
>      processed using line-oriented protocols.  HTTP message  
> generation,
>      in contrast, is usually performed at high speed, encloses data
>      that cannot be modified without also altering its metadata, and
>      is processed using length-delimited protocols.

--
Mark Nottingham     http://www.mnot.net/

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 11 Mar 2008 12:06:13 +0100

Mark Nottingham wrote:

> Overall, I'm not intensely happy with this

>From my POV "showstopper" is a fair summary, see
<http://www.w3.org/QA/2008/03/html-charset.html>

 Frank

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 26 Mar 2008 00:15:38 +1100

Trying to summarise, I think there are two separable issues here;

* 1. Should HTTPBIS continue to accommodate historical clients that  
assume that an unlabeled text/* type is iso-8859-1, rather than MIME's  
default ASCII?

Roy has argued that this is an important distinction that we should  
continue to make, as otherwise existing implementations will become  
non-conformant. Frank points out that no such implementations are in  
common use today, and that those implementations which did make this  
assumption have greater problems (e.g., lacking Host headers).

It would be good to hear if anyone else has an opinion, especially if  
they have experience with / information about such clients, or content  
which relies upon this default.

The conservative thing to do seems to be to keep the status quo. If we  
do that, rather than just close the issue as WONTFIX, we could modify  
the current text to clarify the defaulting (the original question was  
one of precedence between HTTP defaulting and that defined by the  
media type in question), and perhaps give a bit of the history.

* 2. Should HTTPBIS countenance sniffing for character set on text/*  
types?
    a. ...when the charset parameter is not present?
    b. ...when the charset parameter is iso-8859-1
    c. ...at other times?

A few people have noted a security issue in a widely-used browser that  
requires (b). However, I haven't seen a reference to a vulnerability  
report, etc. yet; is anyone aware of one?

Some people have spoken in favour of (a), but I note with interest  
this text in p3, 3.1.1;
> Some HTTP/1.0 software has interpreted a Content-Type header without  
> charset parameter incorrectly to mean "recipient should guess."  
> Senders wishing to defeat this behavior MAY include a charset  
> parameter even when the charset is ISO-8859-1 ([ISO-8859-1]) and  
> SHOULD do so when it is known that it will not confuse the recipient.

If we allow either (a) or (b), this will have to be re-worked.

Also, it's notable that allowing (a) may make (1) easier to resolve in  
favour of dropping the HTTP-specific default; i.e., the default would  
shift from iso-8859-1 to "sniff".

Does anyone think that these are so intertwined that (2) should not be  
a separate issue? If we can resolve it, (1) should follow.

Cheers,

---

From: Simon Perreault <[email-masked]>
Date: Tue, 25 Mar 2008 09:38:40 -0400

Hi Mark,

> assume that an unlabeled text/* type is iso-8859-1, rather than MIME's

Following your comment on a related issue in the vcarddav WG, I investigated 
and came to the conclusion that MIME doesn't specify that text/* has a 
default character set of ASCII. I may very well be wrong, and this email is 
also a way to ask you for clarifications.

I see the definition of the charset parameter in RFC 2046, section 4.1.2. It 
seems specific to text/plain, not text/*.

   A critical parameter that may be specified in the Content-Type field
   for "text/plain" data is the character set.

It is also mentioned that other text/* types may not even have the charset 
parameter. So a default value would be meaningless.

   The specification for any future subtypes of "text" must specify
   whether or not they will also utilize a "charset" parameter, and may
   possibly restrict its values as well.

What I understand from this section is that there is no default character set 
for text/*.

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 25 Mar 2008 17:21:46 +0100

> report, etc. yet; is anyone aware of one?

The security issue is only beause some UAs do sniff making them parse
the content in a different character set than others..

If we do say someting more about this we should very cleary state that
IF the server has indicated a charater set the client MUST use this
characterset and MUST NOT attempt to sniff to guess the characterset.

This is already said in "Missing Charset", kind of.. what is left open
there is how the UA should act if it doesn't support the indicated
characterset..

I prefer to leave it at status-quo on how user agents should behave if
no characterset parameter is specified. The specs is clear that this
means ISO-8859-1, but note that some do things different..

Having ISO-8859-1 as a default instead of US-ASCII is not really a such
big concern as US-ASCII is a pure subset of ISO-8859-1. Content which
was meant to be rendered using US-ASCII will render just fine using
ISO-8859-1.

This said I am in favor of softly depreating the ISO-8859-1,
recommending servers to not rely on it as some user agents is known to
guess the characterset, and therefore recommend to always specify the
characterset explicit when known. But keeping ISO-8859-1 as the official
default if charset is not specified for historic reasons. This means
moving some of the compatibility notes about HTTP/1.0 user agents not
understanding content-type specifications with parameters to historic
notes.

Regards
Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 25 Mar 2008 19:41:24 +0100

Simon Perreault wrote:
 
> I investigated and came to the conclusion that MIME doesn't
> specify that text/* has a default character set of ASCII.
> I may very well be wrong, and this email is also a way to
> ask you for clarifications.

RFC 2045 chapter 5:
| For example, the "charset" parameter is applicable to any
| subtype of "text"

RFC 2045 chapter 5.2:
|    Content-type: text/plain; charset=us-ascii
|
| This default is assumed if no Content-Type header field is
| specified.

RFC 2046 chapter 4.1.2:
| Note that the character set used, if anything other than
| US-ASCII, must always be explicitly specified in the 
| Content-Type field.

There are additional remarks about treating unknown text/*
subtypes like text/plain, and for text/plain the default
US-ASCII is very clear (various places including RFC 2049,
minimal MIME conformance).

You can argue that text/html is not "unknown" for a typical
HTTP application.  *Historic* info:

RFC 2070, chapter 2.1:
| HTML, as an application of SGML, does not directly address
| the question of the external character encoding. This is
| deferred to mechanisms external to HTML, such as MIME as
| used by the HTTP protocol or by electronic mail.
[...]
| Similarly, if HTML documents are transferred by electronic
| mail, the external character encoding is defined by the 
| "charset" parameter of the "Content-Type" MIME header field
| [RFC2045], and defaults to US-ASCII in its absence.

It's fun to see how RFC 2070 avoids to mention the Latin-1
default in RFC 2068 :-)

> It is also mentioned that other text/* types may not even
> have the charset parameter. So a default value would be 
> meaningless.

Yes, I'm too lazy to dig in the IANA registry, are you aware
of a text/* subtype with this property ?

> What I understand from this section is that there is no 
> default character set for text/*.

Not the only possible interpretation, but without doubt the 
dubious text/* MIME default is *NOT* Latin-1, for text/html
in RFC 2070 it is also *NOT* Latin-1.  RFC 2854 is clearer:

RFC 2854 chapter 6 (about text/html):
| The use of an explicit charset parameter is strongly 
| recommended.  While [MIME] specifies "The default character
| set, which must beassumed in the absence of a charset
| parameter, is US-ASCII."  [HTTP] Section 3.7.1, defines
| that "media subtypes of the 'text' type are defined to have
| a default charset value of 'ISO-8859-1'".  Section 19.3 of
| [HTTP] gives additional guidelines.  Using an explicit
| charset parameter will help avoid confusion.

 Frank

---

From: Martin Duerst <[email-masked]>
Date: Wed, 26 Mar 2008 18:19:16 +0900

As for digging on this issue, here a few more quotes from RFC 2070
(Internationalization of the Hypertext Markup Language).
[I was a co-author, but that was a long time ago.]

 From the Introduction:

   The specific issues addressed are the SGML document character set to
   be used for HTML, the proper treatment of the charset parameter
   associated with the "text/html" content type and the specification of
   some additional elements and entities.

1.2.2. User agents

   In addition to the requirements of RFC 1866, the following
   requirements are placed on HTML user agents.

      To ensure interoperability and proper support for at least ISO-
      8859-1 in an environment where character encoding schemes other
      than ISO-8859-1 are present, user agents MUST correctly interpret
      the charset parameter accompanying an HTML document received from
      the network.

      ...

[At the time, there were some user agents that fell over when there
was any charset parameter in a HTTP Content-Type: header at all;
the above was written to make sure that user agents that followed
RFC 2070 would avoid just falling over. My understanding is that
this problem was corrected in version 3 or so of Netscape and IE,
or anyway in a timeframe that makes in irrelevant for our current
spec.]

2.1. Reference processing model

   ...

   For the HTTP protocol [RFC2068], the external character encoding is
   indicated by the "charset" parameter of the "Content-Type" field of
   the header of an HTTP response. For example, to indicate that the
   transmitted document is encoded in the "JUNET" encoding of Japanese
   [RFC1468], the header will contain the following line:

   Content-Type: text/html; charset=ISO-2022-JP

   ...

   Similarly, if HTML documents are transferred by electronic mail, the
   external character encoding is defined by the "charset" parameter of
   the "Content-Type" MIME header field [RFC2045], and defaults to US-
   ASCII in its absence.

[The US-ASCII default is specifically for email; there is no default
 in RFC 2070 for HTTP, for various good reasons, please see separate mail.]

   ...

6. External character encoding issues

   Proper interpretation of a text document requires that the character
   encoding scheme be known.  Current HTTP servers, however, do not
   generally include an appropriate charset parameter with the Content-
   Type header.  This is bad behaviour, which is even encouraged by the
   continued existence of browsers that declare an unrecognized media
   type when they receive a charset parameter.  User agent
   implementators are strongly encouraged to make their software
   tolerant of this parameter, even if they cannot take advantage of it.

[the (hopefully historical) issue mentioned above]

   ...

   In the case where a document is accessed from a hyperlink in an
   origin HTML document, a CHARSET attribute is added to the attribute
   list of elements with link semantics (A and LINK), specifically by
   adding it to the linkExtraAttributes entity.  The value of that
   attribute is to be considered a hint to the User Agent as to the
   character encoding scheme used by the resource pointed to by the
   hyperlink; it should be the appropriate value of the MIME charset
   parameter for that resource.

[not sure how much this is implemented or in use; it's not directly
a HTTP issue]

   In any document, it is possible to include an indication of the
   encoding scheme like the following, as early as possible within the
   HEAD of the document:

    <META HTTP-EQUIV="Content-Type"
     CONTENT="text/html; charset=ISO-2022-JP">

   This is not foolproof, but will work if the encoding scheme is such
   that ASCII-valued octets stand for ASCII characters only at least
   until the META element is parsed.

[This is very, very widely used. As far as it's HTML, it's nothing
HTTP should be concerned, but it is highly relevant for HTTP because
it is dead straight against any default on the charset parameter in
HTTP.]

                                      Note that there are better ways
   for a server to obtain character encoding information, instead of the
   unreliable META above; see [NICOL2] for some details and a proposal.

[Yes indeed, servers never tried to get the charset info out of the
document itself, although (as far as I have been told) that was the
purpose for which this variant of the <meta> syntax had been designed for.]

   For definiteness, the "charset" parameter received from the source of
   the document should be considered the most authoritative, followed in
   order of preference by the contents of a META element such as the
   above, and finally the CHARSET parameter of the anchor that was
   followed (if any).

Not everybody likes this, but it has been around for a long time,
as far as I understand, it's respected by most browsers, and
changing it won't make things better (it's just that the other
half of the people involved will be unhappy).

Regards,    Martin.

>| charset parameter will help avoid confusion.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Wed, 26 Mar 2008 19:37:30 +0900

Hello Mark,

Thanks for laying out the issues. I think there is some
relationship, but it's still good to untangle things.

>assumption have greater problems (e.g., lacking Host headers).

I think Roy is right in that there was indeed a distinction between
HTTP and email here. But Frank is right in that in particular for
HTML and XML (not the smallest 'customers' of HTTP to say the least),
such implementations are indeed not in common use as far as I know.

The reason for this is that in contrast to e.g. text/plain,
HTML and XML come with their internal character encoding
indication mechanism, see below.

>report, etc. yet; is anyone aware of one?

The vulnerability I know about is that there are quite a few cases
of charset combinations where a wrong charset label, interpreted at
face value, can lead to problems. This can happen in the context
of cross-site scripting. I created a (essentially harmless)
example somewhere on the W3C Web site (security by obscurity,
but I can dig it up) when I was still at W3C.

[At some point in time, the Apache standard distribution came with
AddDefaultCharset iso-8859-1
and a comment claiming that this would always be a good idea, but
this was fixed, see
http://mail-archives.apache.org/mod_mbox/httpd-cvs/200502.mbox/[email-masked]%3E]

>Some people have spoken in favour of (a),

It depends a lot on what is meant by "sniffing". For HTML and XML,
there are quite well established methods to look inside the document
and find some information about the charset in there; we have to make
sure we write the HTTP spec so that this is allowed. A proposal
circulated earlier on this list somehow went in this direction, but
it mentioned something like "the first 16 bytes" or so, which doesn't
take into account the actual length up to an "encoding" pseudo-attribute
in the XML declaration, much less the length up to a <meta> element
in HTML. While this mechanism relies on bootstrapping the encoding
and therefore cannot handle any completely new encoding methods
(if such encoding methods would be invented), it works very reliably
for practical cases.

So in my view, (a) clearly has to be allowed in as far as it refers to
content types (e.g. HTML, XML) with a well-defined way to indicate
the character encoding internally, and the extraction and usage of
this information.

Overall, in a case such as HTML, the following is a list of the
priorities of charset information as I think they are mostly being
used in browsers, of if not, should be used:

1. Explicit, per-document override by user
   (after the document has been received and looked at,
    always needed as a last resort because sometimes the label
    is wrong, wherever it may come from)
   [no need to talk about this in the HTTP spec]

2. *explicit* external information, the charset parameter on Content-Type

3. *explicit* information internal to the document for media types
   where this well-defined

[4. potentially information from a link that was followed, although
    I don't think this is widely implemented or used]

5. A 'default' setting on the browser side for unlabeled documents.
   In many cases, the purpose of this is to simply indicate the
   charset that is expected in documents that the user is going to
   view most frequently. This may be iso-8859-1 (or actually
   windows-1252) in Western Europe and much of the Americas, but
   by virtue of that charset being used widely in these areas,
   not by virtue of anything in the HTTP spec. It may be
   iso-8859-2 or so in some parts of Eastern Europe. It is
   typically a "guess Japanese encoding" here in Japan, because
   when limited to charsets customarily used with Japanese,
   detecting the actual encoding actually works very well on
   a reasonably-sized document. It may in some cases also
   include a "guessing" option that tries to guess among any
   and all charsets available to the browser (full-fledged
   "sniffing").

In order to be in line with current practice, we have to make
sure that we don't write the spec to disallow 3. [or 4.] or 5.

>but I note with interest this text in p3, 3.1.1;

This was and is not in line with current practice, and should be
replaced.

>> SHOULD do so when it is known that it will not confuse the recipient.

The fact that this is only a SHOULD is due to the fact, mentioned in
my other mail, that older (VERY older, indeed) clients fell over
in the face of a charset parameter.

>If we allow either (a) or (b), this will have to be re-worked.

Yes indeed.

Regards,    Martin.

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 26 Mar 2008 18:04:59 +0100

> [I was a co-author, but that was a long time ago.]

It's still interesting for reconstructions how precisely
the Web and the Internet at large ended up with Unicode,
later UTF-8, now net-utf8 (in essence NFC).  

My private theory how this all happened is that Harald
considered ISO 2022 as a hopeless case after seriously
trying to make it work, and you consider anything where
it is not intuitively possible to use "ü" as broken by
design.  

 [old browsers hating charset parameters]
> My understanding is that this problem was corrected in
> version 3 or so of Netscape and IE, or anyway in a 
> timeframe that makes in irrelevant for our current
> spec.

+1  IIRC IBM Webexplorer still had issues with it, but
HTTP/1.0 browsers not supporting Host: header fields
are irrelevant today.  And we are not updating gopher
type h, simple HTTP (0.9), or similar relics.

>| In the case where a document is accessed from a 
>| hyperlink in an origin HTML document, a CHARSET
>| attribute is added to the attribute list of 
>| elements with link semantics (A and LINK)
[...]
 
> [not sure how much this is implemented or in use; it's
> not directly a HTTP issue]

Yes, no HTTP issue.  I use 
charset="PC-Multilingual-850+euro" in some links, but
I'm not aware of a spider or browser doing anything with
this info.  Which does not mean that it is necessarily a
waste of time - I'm also not aware of UAs looking at say
hreflang=, but I could add some CSS magic for it later.

>|    <META HTTP-EQUIV="Content-Type"
>|     CONTENT="text/html; charset=ISO-2022-JP">
>| 
>|   This is not foolproof, but will work if the encoding
>|   scheme is such that ASCII-valued octets stand for 
>|   ASCII characters only at least until the META element
>|   is parsed.
 
> [This is very, very widely used. As far as it's HTML,
>  it's nothing HTTP should be concerned, but it is highly
>  relevant for HTTP because it is dead straight against
>  any default on the charset parameter in HTTP.]

Wait a moment, it is dead straight against any default that
is *NOT* ASCII, or rather against a default not containing
ASCII as proper subset.  

For the [i20] question it only tells us that we cannot pick
say BOCU-1 as new default, even if that's MIME compatible. 

Arguably it also tells us that the "default" does not mean
much for HTTP.  It is interesting for HTTP header fields.

For the text/* [i20] issue we might be free to pick ASCII
instead of Latin-1 if that's better for MIME compatibility,
especially for text/plain, naturally for text/xml, and no
problem for text/html.

>| see [NICOL2] for some details and a proposal.

What was NICOL2, was that your heuristic to "sniff" UTF-8 ?
The main problem I have with the "Latin-1 default" is that
it blocks a future "UTF-8 default" (talking about HTTP/1.1)

 Frank

---

From: Martin Duerst <[email-masked]>
Date: Thu, 27 Mar 2008 12:06:43 +0900

At 02:04 08/03/27, Frank Ellermann wrote:
>Martin D$B—S(Bst wrote:

>is *NOT* ASCII, or rather against a default not containing

I think we have to be careful what a HTTP default means.
What a US-ASCII default on the HTTP level means is essentially
that whenever I get something like:
Content-Type: text/foo
I should treat this exactly as if I got:
Content-Type: text/foo; charset=US-ASCII

Now the later means: This is US-ASCII, nothing less and nothing
more. Given that, the browser won't look inside the document
anymore for any additional information. But this is not what
happens in practice, and not what we want.

It could be that by default above, you meant: something
to go back to if *all* else fails. That would mean that
the "default" is only applied if there is no other
information anywhere available about the encoding.
If that's what we want to say, simply saying "default"
is definitely not good enough. And I doubt it is actually
what happens in practice, because if no information is
found, it's usually the browser menu setting (based
on the browser's user interface language or the user's
choice) that kicks in, before a final default has any
chance to be used.

>much for HTTP.  It is interesting for HTTP header fields.

If "default" doesn't mean much, then we shouldn't call
it default.

>especially for text/plain, naturally for text/xml, and no

Many (including me) are advising against text/xml, because
at least according to the books, the US-ASCII default for
text/xml is supposed to be a real default, i.e. there is
no chance to have an internal encoding information work for
text/xml.

>it blocks a future "UTF-8 default" (talking about HTTP/1.1)

There is no need for such a default. Practice already works
without a default.

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 27 Mar 2008 12:25:29 +0900

>later UTF-8, now net-utf8 (in essence NFC).  

Ok. This mail contains the historic bits of the answers
to your mail.

>it is not intuitively possible to use "$B—(B as broken by
>design.  

I don't know about Harald. I don't think that the Umlaut
in my name had too much to do with my interest in
internationalization. I still use a mailer where I cannot
write my name correctly.

It's much more that I was always fascinated by foreign scripts,
and when I saw the first information about Unicode, that seemed
just the obviously right thing to do, listing up all the characters
of the world and giving them numbers. Before I got involved with
internet/WWW internationalization, I had internationalized
ET++, an application framework famous for its contributions
to object-oriended design patterns. I never tried ISO 2022
except for implementing conversion from iso-2022-jp to
Unicode for practical needs. I already knew Francois Yergeau
from the ET++ work, and I knew Glenn Adams from some Unicode
conferences, and then Gavin Nicol came out with a paper that
pointed out that if the Web is going to be one global application,
we better make sure we know what characters we deal with on it.

The biggest problem with iso 2022, in a very big perspective,
is not that operations such as string concatenation are hopelessly
complex. The main problem is that you have data in all these
different codepages that you switch around, and in many cases,
it's the same characters for a user (think about German written
in iso-8859-1 vs. written in iso-8859-2, even the bytes will
be the same in this case), but there is nothing in the iso 2022
architecture that tells you that it's actually the same data.

Gavin put this together with the problem of figuring out what
SGML numeric character references should mean in an internationalized
context, and came up with the idea that they should just mean
Unicode numbers. When he started explaining that on the IETF
HTML WG list, in terms of "SGML document character set" and
such, it seemed pretty much obvious to me (but not too many
others at the time), so then together with Francois and Glenn,
we worked on RFC 2070.

>What was NICOL2, was that your heuristic to "sniff" UTF-8 ?

I don't remember. It definitely wasn't the UTF-8 heuristic,
I started to understand that UTF-8 was extremely easy to detect
in the discussion about FTP internationalization (now RFC 2640)
and then worked through some examples and details and published
them as http://www.ifi.unizh.ch/mml/mduerst/papers/PDF/IUC11-UTF-8.pdf.

My guess is that it was some description of either a scheme to use
extensions to indicate encodings in file names, or something akin to
Apache asis (http://httpd.apache.org/docs/2.2/mod/mod_asis.html), i.e.
a file including all the HTTP headers to be sent.

Regards,    Martin.