Subject: [EAI] UTF8REPLY

From: Shawn Steele <[email-masked]>
Date: Tue, 11 Jan 2011 21:11:52 +0000

If we’re adding it to MAIL FROM: can we change it to just “UTF8” instead of “UTF8REPLY”?  (Eg: I’d like a consistent term that works for MAIL FROM, VRFY and EXPN, and is the same for all 3.

- Shawn

 
http://blogs.msdn.com/shawnste

---

From: [email-masked]
Date: Tue, 11 Jan 2011 13:18:18 -0800 (PST)

> If we’re adding it to MAIL FROM: can we change it to just “UTF8” instead of “UTF8REPLY”?  (Eg: I’d like a consistent term that works for MAIL FROM, VRFY and EXPN, and is the same for all 3.

+1

				Ned

---

From: John C Klensin <[email-masked]>
Date: Tue, 11 Jan 2011 16:36:13 -0500

--On Tuesday, January 11, 2011 21:11 +0000 Shawn Steele
<[email-masked]> wrote:

> If we're adding it to MAIL FROM: can we change it to just
> "UTF8" instead of "UTF8REPLY"?  (Eg: I'd like a
> consistent term that works for MAIL FROM, VRFY and EXPN, and
> is the same for all 3.

FWIW, I'd rather use a parameter name that matches (is identical
to) whatever we turn UTF8SMTPbis into.  Just one for all three
commands, but "UTF8" causes all of the problems that Dave,
JianKang, myself, and others have been discussing -- confusion
of encoding with character repertoire, character repertoire that
doesn't exactly match what the keyword seems to mean, etc.
Since we don't need more than one parameter name, having it
match the EHLO reply value lowers the possibility of confusion
and simplifies things for everyone, IMO.

    john

---

From: Shawn Steele <[email-masked]>
Date: Tue, 11 Jan 2011 21:41:11 +0000

I'm fine if it matches EHLO.  IMO the repertoire isn't really that confusing :)

-----Original Message-----
From: John C Klensin [mailto:[email-masked]] 
Sent: Pōʻ, Ianuali 11, 2011 1:36 PM
To: Shawn Steele; [email-masked]
Subject: Re: [EAI] UTF8REPLY

--On Tuesday, January 11, 2011 21:11 +0000 Shawn Steele <[email-masked]> wrote:

> If we're adding it to MAIL FROM: can we change it to just "UTF8" 
> instead of "UTF8REPLY"?  (Eg: I'd like a consistent term that works 
> for MAIL FROM, VRFY and EXPN, and is the same for all 3.

FWIW, I'd rather use a parameter name that matches (is identical
to) whatever we turn UTF8SMTPbis into.  Just one for all three commands, but "UTF8" causes all of the problems that Dave, JianKang, myself, and others have been discussing -- confusion of encoding with character repertoire, character repertoire that doesn't exactly match what the keyword seems to mean, etc.
Since we don't need more than one parameter name, having it match the EHLO reply value lowers the possibility of confusion and simplifies things for everyone, IMO.

    john

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 12 Jan 2011 08:36:42 -0800

On 1/11/2011 1:11 PM, Shawn Steele wrote:
> If we’re adding it to MAIL FROM: can we change it to just “UTF8” instead of
> “UTF8REPLY”? (Eg: I’d like a consistent term that works for MAIL FROM, VRFY and
> EXPN, and is the same for all 3.

+1

(And using UTF8 does correctly assert that the option covers Unicode encoded in 
UTF8...)

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jan 2011 12:24:38 -0500

--On Wednesday, January 12, 2011 08:36 -0800 Dave CROCKER
<[email-masked]> wrote:

> (And using UTF8 does correctly assert that the option covers
> Unicode encoded in UTF8...)

And incorrectly implies that all of Unicode is permitted as long
as it is encoded in UTF-8.

If we want precision in the use of terms like "UTF-8" (which, by
the way, is the correct spelling), then we should be precise.

    john

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 13 Jan 2011 09:39:28 -0800

> the way, is the correct spelling), then we should be precise.

Choosing labels is often a matter of trade-offs.  There is always a danger of 
the classic perfect being the enemy of the good.

Given that established IETF use of "ASCII" does not typically mean pure ASCII 
and given that the EAI working group and most other IETF use of UTF-8 means 
UTF-8 adapted for use within IETF protocols, the lack of "precision" that you 
cite does not seem to cause all that much trouble.

In fact, there appears to be a similar comfort with approximation within the 
Unicode community.  After the group's previous exchange about the term "Latin" 
and your concerns:

On 1/4/2011 11:09 AM, John C Klensin wrote:
 > Second, while you said "Basic Latin", Dave's revised chart said
 > "Latin".  In Unicode-speak, "Latin" refers to a script (or
 > script family).  That script, according to Section 7.1 of
 > Unicode 5.0, includes the letters of Basic Latin and the Latin-1
 > supplement; the characters of Latin Extended-A, B, C, and D,
 > Extended Additional, and Ligatures.

I happened to find:

    <http://www.unicode.org/charts/>

Which interestingly has an entry under "European Scripts" labeled "Latin".  That 
entry points to:

    <http://www.unicode.org/charts/PDF/U0000.pdf>

which is the very C0 Controls and Basic Latin table we want to reference.  (The 
Label on the chart 'home' page does have the sub-entries you cite, but the label 
itself points only to this one page.

In other words, the Unicode Consortium is comfortable calling the table "Latin".

In any event, since you object to the offered label, presumably you have a 
better term to suggest, so that the working group can make some progress?

d/

-- 

   Dave Crocker

---

From: John Levine <[email-masked]>
Date: 13 Jan 2011 18:03:20 -0000

>better term to suggest, so that the working group can make some progress?

UTF8ISH ?

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jan 2011 15:51:21 -0500

--On Thursday, January 13, 2011 18:03 +0000 John Levine
<[email-masked]> wrote:

>> In any event, since you object to the offered label,
>> presumably you have a  better term to suggest, so that the
>> working group can make some progress?
> 
> UTF8ISH ?

I'm inclined to appeal to minimization of keywords and to
precedent.  The former is important because if we end up with a
situation in the future in which there are lots of parameters
(there are very few today), being able to associate the
parameter name with the relevant extension keyword may actually
be more important than some intuition about meaning.

If you examine the registered extension list and corresponding
EHLO keywords in the mail parameter registry
(http://www.iana.org/assignments/mail-parameters) and then track
down the extension that have corresponding keywords on the MAIL
or RCPT commands, it appears that our general practice has been
the one that I consider most obvious: unless multiple parameter
keywords are needed on a given command to convey distinct
meaning, we simply reuse the EHLO reply keyword name as the
parameter.  

In other words, we set the value for the VRFY/EXPN and MAIL
command parameters to UTF8SMTPbis (whatever that turns out to
be).

Another way to make progress involves actually reading each
other's postings -- this idea has not only been mentioned
before, but more than one person has expressed support for it.

    john

---

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jan 2011 16:11:13 -0500

--On Thursday, January 13, 2011 09:39 -0800 Dave CROCKER
<[email-masked]> wrote:

>...
> Given that established IETF use of "ASCII" does not typically
> mean pure ASCII and given that the EAI working group and most
> other IETF use of UTF-8 means UTF-8 adapted for use within
> IETF protocols, the lack of "precision" that you cite does not
> seem to cause all that much trouble.

Dave, you raised this issue in commenting on the use of ASCII
and non-ASCII, which, for better or worse, is also extremely
established IETF usage.  I've agreed with you that the usage is
worth changing, but, if we are going to do it, let's actually do
it, not complain first and then suggest making the same errors
again.

> In fact, there appears to be a similar comfort with
> approximation within the Unicode community.  After the group's
> previous exchange about the term "Latin" and your concerns:

> Which interestingly has an entry under "European Scripts"
> labeled "Latin".  That entry points to:
> 
> which is the very C0 Controls and Basic Latin table we want to
> reference.  (The Label on the chart 'home' page does have the
> sub-entries you cite, but the label itself points only to this
> one page.

> In other words, the Unicode Consortium is comfortable calling

Actually, the comfort within the Unicode community is that, as
the Standard, and text that describes it, has evolved
contradictions have crept in.  Virtually everyone who have been
working with i18n issues for any length of time is aware of
them, interprets intended meaning from context, and moves on.
This is one such example, where "Latin" is used to refer to the
ASCII code space, and the letters within that code space, and
the graphic (printing) characters within that code space, and
the whole collection of Latin-derived characters and marks that
are associated with the script.   Part, but only part, of the
problem is that it is common to use "Latin" to refer to "Latin
characters" and "Latin script" (without the qualifications) but
sometimes referring to somewhat different repertoires.

There are really only two ways around these problems:

	(1) Use the somewhat-imprecise terminology, accept the
	ambiguity and occasional need to deduce what is
	intended, and promise never to complain about
	low-precision or ambiguous terminology.
	
	(2) Learn to be very precise.  Because the list of
	characters associated with a script or such properties
	as "letter", "digit", "format character", etc. (see RFC
	3536 for some of this) even defining those in terms of
	blocks or ranges is imprecise unless the particular
	version of Unicode is specified.  Consequently, being
	precise typically requires either inventing completely
	new terms or building definitions based on stable
	Unicode properties, not blocks or names for groups of
	code points.   FWIW, IDNA2008 ended up needing to do
	both, as Andrew pointed out some time ago.

The WG effectively selected the first approach for its
Experimental documents and received no negative comments about
it during Last Call on those documents.   Your review of 5336bis
caught a few actual errors --which I, at least, appreciate-- but
raised this more general issue of precision as a serious
problem.  My inclination is that we are better off fixing that
problem.  But I think that, if we are going to do that, being
consistent about it has some merit.

     john

   john

---

From: Shawn Steele <[email-masked]>
Date: Thu, 13 Jan 2011 21:24:40 +0000

I disagree as to the precision of the meaning :)  UTF-8 is often used for contexts where the full range of code points isn't legal, or where there are other special rules.  HTML for example can be declared as UTF-8, but that doesn't mean I can use any code point I feel like, particularly in any order I feel like, there are other rules that restrict which parts of Unicode can be used in what places.

And, IMO, it's a bit easier to remember UTF8 as a keyword for globalized than some abstract term.  UTF8 also has the advantage of impressing that it is clearly UTF-8, and you aren't to abuse this standard with some other encoding.

-Shawn

-----Original Message-----
From: John C Klensin [mailto:[email-masked]] 
Sent: Poʻahā, Ianuali 13, 2011 9:25 AM
To: [email-masked]; Shawn Steele
Cc: [email-masked]
Subject: Re: [EAI] UTF8REPLY

--On Wednesday, January 12, 2011 08:36 -0800 Dave CROCKER <[email-masked]> wrote:

> (And using UTF8 does correctly assert that the option covers Unicode 
> encoded in UTF8...)

And incorrectly implies that all of Unicode is permitted as long as it is encoded in UTF-8.

If we want precision in the use of terms like "UTF-8" (which, by the way, is the correct spelling), then we should be precise.

    john