Subject: [EAI] First comments on draft-klensin-net-utf8-oo.txt

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 03 May 2006 15:14:52 +0100

This seems mostly good, as setting the lowest level of what should be  
allowed to pass over the internet. Naturally, individual protocols may be  
more restrictive (e.g. by requiring stronger normalization).

What is not clear is whether recipient agents can assume, without  
checking, that what they receive is compliant to this spec (i.e. anything  
that arrived that was not conformant would result in "garbage in - garbage  
out". It seems to be the intent of the Unicode people (according to their  
"early normalization" principle) that such assumptions could be made. For  
example, do current agents, in general, check for strict conformance with  
NVT?

The other matter is what to do about future versions of Unicode. My  
suggestion is that it would be better to say that code points classed as  
"Not assigned" MUST NOT be used. That means "Not assigned" in the version  
of Unicode that the generator of the text is using, and hence it means  
that he MUST use the definition of NFC contained in that same version.  
Since this spec is essentially to be observed by those who _generate_  
texts, observance of this rule would be sufficient to prevent  
un-NFC-normalized texts appearing on the internet even for NFC  
normalizations not yet invented.

This, of course, assumes that the Unicode people do not alter the  
normalizations for any code point already assigned, but they seem pretty  
good at avoiding such pitfalls. It would still allow them to invent new  
normalizations when new code points were assigned.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: John C Klensin <[email-masked]>
Date: Wed, 03 May 2006 13:52:35 -0400

--On Wednesday, 03 May, 2006 15:14 +0100 Charles Lindsey
<[email-masked]> wrote:

> This seems mostly good, as setting the lowest level of what
> should be  allowed to pass over the internet. 

Well, that is not quite the intent.  And it is _not_ a work item
for the EAI WG.

> Naturally,
> individual protocols may be  more restrictive (e.g. by
> requiring stronger normalization).
> 
> What is not clear is whether recipient agents can assume,
> without  checking, that what they receive is compliant to this
> spec (i.e. anything  that arrived that was not conformant
> would result in "garbage in - garbage  out". It seems to be
> the intent of the Unicode people (according to their  "early
> normalization" principle) that such assumptions could be made.
> For  example, do current agents, in general, check for strict
> conformance with  NVT?

I think the robustness principle necessarily applies. For
protocols that specify this form, recipient agents can
rationally assume that they form is in use, but still need to
protect themselves against serious violations.  I would expect
better-quality implementations to go beyond that in what they
accept, especially if they are sent trash often enough that
market forces drive them in that direction.

> The other matter is what to do about future versions of
> Unicode. My  suggestion is that it would be better to say that
> code points classed as  "Not assigned" MUST NOT be used. That
> means "Not assigned" in the version  of Unicode that the
> generator of the text is using, and hence it means  that he
> MUST use the definition of NFC contained in that same version.
> Since this spec is essentially to be observed by those who
> _generate_  texts, observance of this rule would be sufficient
> to prevent  un-NFC-normalized texts appearing on the internet
> even for NFC  normalizations not yet invented.

Note that, if UTC never adds another precomposed character to
the collection, which they say they have promised to do, then
there will never be a new NFC normalization.  More broadly, if
this specification applies to general-purpose text, rather than
something that we can restrict further for good reason within a
particular protocol, I think it is plausible to assume that
people will send whatever codes they need, for whatever
characters they need, from whatever version of Unicode is
current (or current enough to contain those characters).  Saying
"don't do it" will almost certainly be ignored, and we are
better off designing under that assumption.

> This, of course, assumes that the Unicode people do not alter
> the  normalizations for any code point already assigned, but
> they seem pretty  good at avoiding such pitfalls. It would
> still allow them to invent new  normalizations when new code
> points were assigned.

See above.

    john

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 May 2006 09:20:00 -0400

See above.

    john