Subject: Re: [EAI] Comments on draft-ietf-eai-downgrade-02

From: [email-masked]
Date: Fri, 20 Oct 2006 22:08:25 +0900 (JST)

Thanks for your comment Charles.
I'm sorry for this late reply. I was read, but then, missed this.

I try to update the draft before cut-off, if possible.

> From: "Charles Lindsey" <[email-masked]>
> Date: Mon, 04 Sep 2006 20:21:38 +0100
> 
> s/EAI/UTF8SMTP/ throughout the document, including in the title.

I'm hesitating EAI or UTF8SMTP.
framework document is 'EAI Framework'.
Which is this protocol set name, EAI or UTF8SMTP ?

> 1.  Introduction
> 
>     Traditional mail systems which are defined by [RFC2821] and [RFC2822]
>     allow US-ASCII characters in SMTP envelope and mail headers in body
>     part.  The EAI proposal [I-D.ietf-eai-framework],
>     [I-D.ietf-eai-utf8headers] and [I-D.ietf-eai-smtpext] allows UTF-8
>     characters in SMTP envelope and mail headers in body part.
> 
> s/mail headers in body part/mail header field bodies/ (twice)

agreed and updated.

> otherwise, people will think you are talking about headers inside the  
> <body>s of mail messages. "field body" is the term RFC 2822 uses for a  
> header field without its header name.
> 
>     ... Not only
>     SMTP envelope, but also UTF-8 characters in mail headers MUST be
>     converted to US-ASCII.
> 
> s/mail headers/mail header fields/ here and in lots of other places, to  
> accord with the terminology of RFC 2822.

agreed and updated.

> And then s/mail header fields/mail header fields and in the headers fields  
> of MIME multipart and message types/.
> 
> Otherwise, an already downgraded message will get munged if it then  
> encounters a non-8BITMIME server.

not agreed. An already downgraded message is a normal rfc2822/rfc822
message and it may contain 8bit(non-ascii) characters in the message
body. RFC 1652 capable SMTP agent treats it correctly.

>     In this document, requirements for downgrading is described in
>     section Section 3, SMTP Downgrading is described in Section 4, and
>     SMTP DATA/Header Downgrading is described in Section 5.
> 
> s/is described/are described/

updated.

> I don't like the use of the term "SMTP DATA/header" to describe the mail  
> messages that are transported. We are speaking of objects that conform to  
> RFC 2822 as modified by our utf8headers document, but these objects are  
> going to turn up in all sorts of places apart from within SMTP, such as  
> within other transports (e.g. UUCP), in gateways from other media (e.g.  
> News), within mailing list expanders, and within POP3, IMAP and other  
> storage agents, and they are likely to have to be downgraded (or upgraded)  
> according to your rules in Section 5 in all sorts of situations (e.g.  
> various gateways). Of course, your Section 4 is SMTP-specific, but not  
> your Section 5.
> 
> So a better term would be "Email header downgrading".

agreed.

> 3.  Downgrade Requirements
> 
> 3.1.  Timing and conditions of downgrading
> 
>     Note: If the UTF8SMTP header exists, downgrading will be performed.
>     If UTF-8 characters exist in mail headers without the UTF8SMTP
>     header, this is a protocol error, ...
> 
> No, only if that header exists with the parameter "UTF8", not when its  
> parameter is "downgraded" or "ASCII".

agreed.

> Note also that I have suggested renaming this header as something like  
> "Header-Content", and using "UTF8SMTP" as the parameter.

to make uniform with utf8header document change,
Header name changed to "Header-Type" and the parameter value to "UTF8".

> 3.2.  Requirements
> 
>     Followings are requirements of downgrading.
> 
> I think that is more a list of hoped-for properties rather than a list of  
> requirements. I doubt we are going to succeed in realizing them all (e.g.  
> an upgrade may not always be an exact inverse of the original downgrade).

changed some items' "must" to "should".

> 4.  SMTP Downgrading
> 
>     Downgrading in SMTP envelope uses ALT-ADDRESS parameter proposed in
>     [I-D.ietf-eai-smtpext].  An address is downgradable if the address is
>     all-ASCII address or the address has all-ASCII address specified by
>     ALT-ADDRESS parameter.
> 
> But what about punycode downgraded (possibly under the control of an  
> "atomic" or "downgradable" keyword)? It is clear that such a downgrade is  
> achievable, given the offficial definition or punycode. We discussed that,  
> but we never agreed to remove it from the draft AKAIR.

At the Vancouver discussion, MUA or submission server computes or
specifies ALT-ADDRESS parameter if necessary.

>     One SMTP session may contain multiple recipients.  Downgrading SHOULD
>     be performed for each SMTP recipient address.  ...
> 
> s/for each SMTP recipient address/for each SMTP recipient address  
> individually/

updated

>     EAI-Downgraded-From:  <Non-ASCII {US-ASCII}> <US-ASCII>
>     EAI-Downgraded-To:  <Non-ASCII {US-ASCII}> <US-ASCII>
> 
> You need to give a syntax for these new header fields. Also, they should  
> be regarded as Trace fields, so that they come in the middle of the  
> sequence of Received headers, to indicate whereabouts in the sequence of  
> MTAs the downgrading was done.
> 
> Also, we need a name not involving "EAI-", and I would suggest something  
> involving "Envelope-". So we would get something like:
> 
>     trace         =       [return]
>                           1*( received / envelope-down )
>     envelope-down = "Envelope-Downgraded-From:" downchange CRLF /
>                     "Envelope-Downgraded-To:" downchange CRLF
>     downchange    = [CFWS] "<" utf8-addr-spec [alt-address] ">" [CFWS]
>                        "<" [CFWS] addr-spec [CFWS ">" [CFWS]

agreed. I will try to add this.

>     ESMTP ORCPT parameter is used for Delivery Status Notifications
>     (DSNs) [RFC3461].  ... ORCPT
>     parameter downgrading should be defined here.
> 
> Then please let us have a specific proposal for the next draft.

It depends on ORCPT extension document.

> 5.  SMTP DATA/Header Downgrading
> 
> As I said earlier, this needs a title that does not confine it to things  
> that appear in an SMTP DATA command.
> 
>     Originator address(es):  ...
>     Destination address(es):  ...
>     IDs:  IDs such as Message-ID:, Date:, In-Reply-To: and References: ...
>     Trace headers:  ...
> 
> You need to add Content-Type, Content-Disposition, etc. to your list,
> and also List-*.

In my opinion, Content-Type, Content-Disposition headers are not
expanded and do not contain UTF-8 characters.  So it is included in
other headers and they are not target of downgrading.

I considered that List-* headers as normal unstructured headers.
It needs more considerations.

> Headers (Content-*) containing <parameter>s will be downgraded according  
> to RFC 2231, and headers containing IRIs (List-*) according to RFC 3987.

It's hard to define all headers' downgrading method. but it is necessary...
Hmm...

Are IRIs required to use IRI specific downgrading?
Is encapsulating in Downgraded header allowed?

In my previous opinion, to ease to implement, conversion target
headers should be minimized, such as: From, Sender, Reply-To, To, CC,
....

Previously I considered another headers are simply encoded in RFC 2047
or encapsulated to Downgraded header. Subject header or some
unstructured headers are encoded in RFC 2047, but most other headers
are not...... It requires more consideration.

> 5.1.  Header conversion
> 
>     To preserve all header information, define generic encapsulation
>     header: "Downgraded: HeaderName: HeaderValue".  The header value is
>     encoded by [RFC2047] with UTF-8 tag.
> 
> Again, some syntax is needed. I would suggest:
> 
>     fields     =/ downgraded
>     downgraded = "Downgraded:" [CFWS] field-name ":" unstructured CRLF
>     field-name = 1*ftext
>     ftext      = %d33-57 / %d59-126 ; printable ASCII except ":"

I will add this.

>     Downgrading:
>        *  Check if the mail has 'UTF8SMTP' header and its value is
>           "UTF8".  Otherwise, downgrading is not required.
>        *  Check each header whether it contains UTF-8 characters or not.
> 
> You need to deal with the RFC 2047 stuff here:

OK,

> |        If the header contains any <unstructured>, or any <commemt>s
> |        or <phrase>es
> |        + encode them according to RFC 2047
> |        If the header contains UTF-8 characters,
> |                     ^
> |                     still
>           +  If the header is an address header which is described in
>              Section 5.1.1,
>              -  Preserve the header in 'Downgraded' header.
> No, that comes a little later
>              -  Downgrade the header defined in Section 5.1.1.
> |        + If the header field contains any <parameter> ... Section 5.1.2  
> ...
> |        + If the header contains any IRI ... Section 5.1.3 ...
>           +  Preserve the header in 'Downgraded' header.
> 
>           +  The other header case, encode the header by [RFC2047] with
>              UTF-8 tag.
> But NO!!! You cannot just apply RFC 2047 to any other header which just  
> happens to contain UTF-8. So far, nobody has defined any UTF-8 headers not  
> already covered above, but for sure they will appear in the future. The  
> only thing you can do with them (though maybe the guy who invented such a  
> header also invented a downgrade for it) is to create a Dowwgraded header  
> for it (which of course can then contain RFC 2047).

agreed, but too many changes.... if possible, ...

And we need to write complete downgradable structured/unstructured header list.
Other headers should be encapsulated in "Downgraded" header, I think.

>        *  Change 'UTF8SMTP' header value to "Downgraded".
> 
> And point out that you don't need to create a Downgraded header field if  
> the only thing you did was to apply RFC 2047 to <unstructured>s,  
> <comment>s and <phrase>s.

agreed.

>     Upgrading:
> 
>           +  If the header is address header described in Section 5.1.1,
>              -  Apply address header downgrading to the decoded header.
> Add "using the Downgraded header field"
>              -  Remove the header line which is the same with the
>                 downgraded line.
> 
> What do you mean by "same"? Suppose the folding if different. Suppose  
> extra whitespace has appeared/disappeared? Suppose some RFC 2047 encoding  
> has been done differently, or with a different character set?
> And what if there isn't any matdhing header? Surely that is a protocol  
> error? Do we want to define what happens in that case (e.g. leave the  
> header in place, in which case you may have two From (or To) headers,  
> which is a breach of RFC 2822)?

This part needs detailed procedures: folding and whitespaces
normalizations, error handlings, ...

>           +  Remove the 'Downgraded' header.
>           +  Add decoded header to mail header.
>        *  If each mail header has [RFC2047] encoded part and which
>           encoding is "UTF-8", it is a downgraded header, so decode it.
> And of course that is one possible cause why the upgraded header may not  
> be identical to the pre-downgraded header.

I agree.

>        *  Change 'UTF8SMTP' header value to "UTF8".
> 
>     Pros:
>        *  EAI incompliant MUA displays the downgraded mail body except
> 
> s/mail body/header field bodies/

updated.

>     Cons:
>           ... Also, restoration order of the
>           downgraded headers is not guaranteed.  ...
> 
> I think you ought to define exactly where the Downgraded headers and  
> modified headers are to be put (e.g. at the same place in the message as  
> the old version). This should be a "SHOULD".

Ok, it is reasonable, I will add.

> 5.1.1.  Downgrading address headers
> 
>     The header value is composed of single or multiple mailbox/angle-addr
>     fields defined in [I-D.ietf-eai-utf8headers].
> 
> s/header value/header field body/

updated.

> The next bit would need to be adapted to the changes I have suggested for  
> 5.1 above.

I will update this part.

>     2.  By mailbox/angle-addr downgrading, if the field became empty, the
>         field should be removed.
>     3.  If all header field is removed, remove the header.
> 
> But suppose the header was
>      From: "Joe Doe" <utf8-joe@utf8-domain>
> You will get
>      From: "Joe Doe"
> which makes it Non-RFC 2822-compliant, but you cannot remove it because it  
> is not empty.

Yes, I will consider this problem.

>     4.  If From header is removed, generate new From: header from
>         envelope-from address.
> 
> Ugh! Though I cannot think of anything better.

This part need more careful consideration.

>     2.  <Non-ASCII>
>            Non-ASCII mail address without ALT-ADDRESS parameter case,
>            remove this angle-addr.
> 
> No, that is not right. If the <angle-addr> is
>      <ascii-joe@utf8-domain>
> then you should be able to translate the utf8-domain with punycode and  
> carry on.

Agreed, but it may need more considerations.

Without ALT-ADDRESS, US-ASCII user-part and UTF-8 domain-part address
may be downgraded in SMTP Downgrading.

May I consider the address which has ascii user-part and utf8
domain-part is US-ASCII mail address and mechanically transformable ?

> And, as I have already said, I would still like to be able to translate  
> the <local-part> using punycode, possible restricted by an  
> atomic/downgradable parameter.
> 
> 8.  IANA Considerations
> 
>     IANA is requested to add the "EAI-Downgraded-From:",
>     "EAI-Downgraded-To:" and "Downgraded:" new headers to the registry
>     with the entries pointing to this specification for its definition.
> 
> No, you have to provide templates for all your new headers, as I explained  
> when commenting on the utf8headers draft.

Ok,

> Appendix A.  Examples
> 
> Some nice esamples here, but it would be better to show them as a  
> continuous series in one Appendix. As it is, each Appendix starts by  
> repeating where the previous Appendix left off, which is wasteful as well  
> as confusing.
> 
>     Message-Id: MESSAGE_ID
> 
> s/MESSAGE_ID/<MESSAGE_ID>/ in all the examples.
> 
>     From: <NON-ASCII-FROM {ASCII-FROM}>
>     To: <NON-ASCII-TO {ASCII-TO}>
>     CC: <NON-ASCII-CC>
> 
> Please can we include some <display-name>s and some cases of multiple  
> <angle-addr>s in the same header in the examples?

Ok, I will update.

Thank you.

--
Kazunori Fujiwara, JPRS

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 20 Oct 2006 20:05:06 +0100

> Which is this protocol set name, EAI or UTF8SMTP ?

I think some gathering at an IETF agreed that the visible name of our  
endeavour should have "UTF8" in it, and the provisional choice was  
"UTF8SMTP" (I would have preferred "UTF8-Headers" myself). I don't think  
the world out there is going to accept the term "EAI" (they will have  
enough trouble remembering what it stands for :-( ).

> body. RFC 1652 capable SMTP agent treats it correctly.

No it doesn't. A part of a multipart mixed has some headers (usually  
Content-Type and Content-Transfer-Encoding) before the body of that part.  
Now, those headers can contain UTF-8 (in comments at least, but also in  
<parameter>s) and RFC 1652 does not know they have to be downgraded.  
Likewise the mail headers in message/rfc822. I am arguing with John about  
this one at the moment, so let us see how that goes.

> It depends on ORCPT extension document.

OK, so long as it doesn't get forgotten.

> other headers and they are not target of downgrading.

They can certainly contain UTF8 in comments (they never do, of course, but  
it is allowed). The more realistic case is when a Content-Disposition  
contains a filename parameter in UTF-8. RFC 2231 is the proper downgrade  
there (ugly, but it works).
> I considered that List-* headers as normal unstructured headers.

No, they are structured. There is ABNF for them, and they mostly contain  
URLs.

> Are IRIs required to use IRI specific downgrading?

There is a standard downgrade to URI, described in RFC 3987. So you just  
have to tell people to use that. But isn't someone in this WG supposed to  
be writing a draft on mailing lists which would cover those headers? One  
would certainly wish IRIs/URIs to be readable by speakers of the relevant  
language. At the moment, a lot of them (URIs) can get horribly opaque,  
even to English speakers.

> Is encapsulating in Downgraded header allowed?

If it were done as an addition (like we are proposing even for address  
headers) it would be fine. You might get away with it as the sole  
mechanism if you assume that these IRIs don't need to be seen until the  
message has been upgraded at the far end. There, people will expect to be  
able to click on them, but they are not of much use during transport.
> or encapsulated to Downgraded header.

Agreed that all headers we do not recognise (and if they contain more than  
comments to be downgraded) will produce the Downgraded header only. That  
works well enough for headers that don't have to be looked at until after  
upgrading at the far end. I am not sure whether that is good enough for  
Content-Disposition, though.

> normalizations, error handlings, ...

I think, if you cannot identify the header that was downgraded in order to  
remove it, then you just have to leave it alone. In any case, some  
scammers are going to make them deliberately different, and that needs to  
be mentioned as a Security Issue

> may be downgraded in SMTP Downgrading.

Correct. It is when there is a utf-8 local-part and no alt-address that we  
have problems. But I suspect that will be the common case for people who  
never expect to be mailing to pure-ascii-users.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: John C Klensin <[email-masked]>
Date: Fri, 20 Oct 2006 23:52:59 -0400

>>> Otherwise, an already downgraded message will get munged if
>>> it then encounters a non-8BITMIME server.
>> 
>> not agreed. An already downgraded message is a normal
>> rfc2822/rfc822 message and it may contain 8bit(non-ascii)
>> characters in the message body. RFC 1652 capable SMTP agent
>> treats it correctly.
> 
> No it doesn't. A part of a multipart mixed has some headers
> (usuallyContent-Type and Content-Transfer-Encoding) before the
> body of that part.Now, those headers can contain UTF-8 (in
> comments at least, but also in<parameter>s) and RFC 1652 does
> not know they have to be downgraded.Likewise the mail headers
> in message/rfc822. I am arguing with John aboutthis one at the
> moment, so let us see how that goes.

We don't have much of an argument.   You are assuming that those
attached MIME headers (C-T and C-T-E) can (and, I assume,
should) be permitted to contain non-ASCII characters).   I am
assuming that is reasonable iff (i) the utf8headers document is
explicit about it (which it had better be, one way or the other)
and (ii) if there is a plan for downgrading those headers that
is clear and reversible without loss of information. 

Clearly, RFC 1652 downgrading cannot apply to such headers -- it
(as you point out) doesn't know and can't be expected to know. 

This is _not_ a proposal, and I hope we don't need to go in this
direction.  However, it occurs to me that one way out of many of
these problems would be to require that the host proposing to
downgrade a message check each header for the presence of 8bit
characters.  Then, if it recognizes each one --knows the header
name and the syntax rules for that header-- it can downgrade the
message.  If not (i.e., if there are any headers containing 8bit
characters that are not explicitly known to the MTA), then the
message must be rejected or an NDN issued.

>>> You need to add Content-Type, Content-Disposition, etc. to
>>> your list, and also List-*.
>> 
>> In my opinion, Content-Type, Content-Disposition headers are
>> not expanded and do not contain UTF-8 characters.  So it is
>> included in other headers and they are not target of
>> downgrading.
> 
> They can certainly contain UTF8 in comments (they never do, of
> course, butit is allowed).

Again, it seems to me that it is allowed only if we decide to
allow it.  Certainly it is not allowed under 2822, which
restricts all headers and all fields, including comments, to
ASCII. 

> The more realistic case is when a
> Content-Dispositioncontains a filename parameter in UTF-8. RFC
> 2231 is the proper downgradethere (ugly, but it works).

Actually, it doesn't work.  In the general case, the downgrading
MTA can't possibly know anything about the syntax of a filename
on the system containing the delivery MUA (not just MTA) -- that
filename is interpreted on the far side of POP or IMAP servers
if they are used.  In particular, as far as it knows, the
encoded word form is a valid filename on the target system _and_
that system supports non-ASCII file names as well (not
necessarily UTF-8 ones).  My guess is that the only possible
downgrade mechanism for a Content-disposition header containing
8bit information is to discard it -- or at least to discard any
parameters with 8bit values.

>...
>>> Headers (Content-*) containing <parameter>s will be
>>> downgraded according to RFC 2231, and headers containing
>>> IRIs (List-*) according to RFC 3987.
>> 
> There is a standard downgrade to URI, described in RFC 3987.
> So you justhave to tell people to use that. But isn't someone
> in this WG supposed tobe writing a draft on mailing lists
> which would cover those headers? Onewould certainly wish
> IRIs/URIs to be readable by speakers of the relevantlanguage.
> At the moment, a lot of them (URIs) can get horribly
> opaque,even to English speakers.

But even knowing that something is an IRI or URI requires either
heuristics (bad news) of knowledge of a particular header and
its syntax.  I don't see how one applies IRI-> URI downgrading
to a string in an arbitrary unknown field.

>...
>>> And, as I have already said, I would still like to be able
>>> to translate the <local-part> using punycode, possible
>>> restricted by an atomic/downgradable parameter.

But that idea was already rejected by the WG.  More important,
whatever you do, you can't use punycode, which is
case-insensitive.  You would need a different ACE.

    john

---

From: Kari Hurtta <[email-masked]>
Date: 21 Oct 2006 08:44:45 +0300

> parameters with 8bit values.

I think that there is little disagreement :-)

If there is

  Content-Type: text/plain
  Content-Disposition: attachment; filename="Ääkkösiä nimessä.txt"

downgrade can produce something like

 Content-Type: text/plain
 Content-Disposition: attachment; filename*0*=UTF-8''%C3%84%C3%A4kk%C3%B6si%C3%A4%20nimess;
    filename*1*=%C3%A4%2Etxt

If MUA does not understand RFC 2231, MUA does not see filename parameter here
(it looks like thare is parameters "filename*0*" and "filename*1*").

If MUA undrestands RFC 2231, there is charset given on parameter. Sure
it is then prepared to convert charset to locally used character set/encoding.

And MUA must be prepared to check syntax of filename also when RFC 2231
is not used (adn there is ASCII only filename). In general MUA which
may allow saving attachments, need anyway (prepared to) adjust given filename.

Converting MTA does not know syntax for filenames used on recipient's MUA,
but sending system either  does not know syntax for filenames used on 
recipient's MUA -- this same happens also with ASCII only filenames.

/ Kari Hurtta

I exactly do not follow RFC 2231 by default here. My MUA
produces also ASCII only filename:

Content-Type: text/plain
Content-Disposition: attachment; filename*0*=UTF-8''%C3%84%C3%A4kk%C3%B6si%C3%A4%20nimess;
 filename*1*=%C3%A4%2Etxt;
 filename="____kk__si__ nimess__.txt"

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 21 Oct 2006 08:11:34 +0200

> restricts all headers and all fields, including comments, to

RFC 2046 is already at DS (unlike 2822), and screwing with MIME 
is dangerous.  The WG charter explicitly mentions mailing lists,
that _could_ cover List-*.

> I don't see how one applies IRI-> URI downgrading to a string
> in an arbitrary unknown field.

Until 2368bis exists let's ignore this as too difficult, folks
can use URLs in header fields, and translate any IRI to an URL.

>>>> I would still like to be able to translate the <local-part>
>>>> using punycode, possible restricted by an atomic/downgradable
>>>> parameter.
 
> But that idea was already rejected by the WG.

When was that ?  I vaguely recall a statement posted by Harald,
and a debate about consensus on the list or elsewhere, with an
outcome of "elsewhere is irrelevant" (?)

> you can't use punycode, which is case-insensitive.  You would 
> need a different ACE.

The author told us that it's not necessarily case-insensitive:

  <http://article.gmane.org/gmane.ietf.ima/438> 
|| the underlying Punycode algorithm will copy case-sensitively
|| perfectly satisfactorily (even using their model implementation
|
| Right.  I designed Punycode to work for case-sensitive text,
| case-insensitive text, and case-insensitive case-preserving text.

Frank

---

From: [email-masked]
Date: Mon, 23 Oct 2006 19:55:08 +0900 (JST)

> enough trouble remembering what it stands for :-( ).

I agree that "EAI" is not good.
It may be "UTF8SMTP".

Working group name is "EAI" and 
protocol/document set name is "UTF8SMTP", is it right?

> this one at the moment, so let us see how that goes.

I understand. It is hard problem.

# I don't like rewriting the body of a message.

As John commented, these MIME headers needs further duscussion and
describing in utf8header document.

> No, they are structured. There is ABNF for them, and they mostly contain  

Ok, downgrading methods are different for each headers.  We need to
list up target header list and arrange them, and define downgrading
methods for each headers.

Some headers which contains IRI only should be downgraded with IRI
specific downgrading method.

Some headers like 'Subject' are simply encoded by RFC 2047.

> able to click on them, but they are not of much use during transport.

I'll add that headers which is not in downgradable header list are
encapsulated in Downgraded header.

> be mentioned as a Security Issue

Ok, to add some lines about:

  If hard to identify to erase, leaving original Downgraded header is
  one solution.  (decoding RFC 2047 encoding increases user's
  readability.)

> never expect to be mailing to pure-ascii-users.

I'll add.

--
Kazunori Fujiwara, JPRS

---

From: [email-masked]
Date: Mon, 23 Oct 2006 20:22:28 +0900 (JST)

> From: [email-masked]

Modifications are many and there are many considerations,
it is hard to submit before today's cut-off.

I'll list considerations and next version's draft to this list next
week.

--
Kazunori Fujiwara, JPRS

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Oct 2006 15:51:44 +0100

> is clear and reversible without loss of information.

My reading of our utf8hdrs document is that UTF8 can be used
     in any header described as <unstructured>
     in <comment>s wherever they are allowed to occur in any header
     in <phrase>s wherever they are allowed to occur in any header
     and wherever else explicitly allowed, as in <addr-spec>s.

As to which headers <comment>s may appear in, RFC 822 has some very
liberal rules which any header defined in RFC 822 terms (and that
certainly includes C-T and C-T-E) is free to apply. RFC 2822 is more
explicit, but in general confirms what RFC 822 says. Some more recent
drafts based on RFC 2822 (notably USEFOR and DKIM) have restricted the
places where <comment>s may appear (by using [FWS] rather than [CFWS]).

It is pretty easy for MUAs to detect where comments are allowed under the
RFC 822 rules, without knowing the exact syntax of individual headers, and
I suspect most current MUAs simply detect them on that basis and display
stuff accordingly (e.g. by recognising encoded-words there).

> characters.

I think that if a downgrader sees a Header-Type: UTF8SMTP at the top
level, and if the C-T is some kind of multipart, then it has to decend
(recursively if necessary) through the multipart structure looking for
Content-* headers with <comment>s or <parameter>s with UTF-8 in them. And
at the same time it needs to look at the headers of any message/rfc822 it
encounters (though checking for the presence/absence of Header-Type:
UTF8SMTP in there should suffice). You might even insist on a Header-Type
header in the introductory heading of such parts (though RFC 2045/6
currently allows only headers starting with "Content-" there).

>> Content-Disposition contains a filename parameter in UTF-8. RFC
> if they are used.

There is no requirement in RFC 2183 for the sending MUA or any
intermediate MTA to avoid filnames which might not be syntactically
acceptable at the receiving end. The 'filename' parameter is just a
'suggestion', and it clearly states that it is up to the receiving MUA to
decide whether it is fit to be used as it stands.

Moreover, RFC 2183 clearly states that, although filenames would be
expected to be in ASCII at the time of writing, it was to be expected that
future enhancements would make provision for Non-ASCII filenames. It might
be argued that RFC 2231 had already provided that possibility, but if not,
then it is clearly the case that our present UTF8SMTP has now provided
precisely the opportunity envisaged. So it is not a question of _whether_
we permit ASCII filenames, but _how_. And a downgrade via RFC 2231 is the
obvious mechanism (though we might encourage some normalization, probably
NFKC, while we are about it). And whatever provision we make should apply
to <parameter>s in general. Naturally, <token>s will remain in ASCII, as
well as values that are keywords (such as the IANA-registered charsets).
Non-ASCII boundaries, anyone?

> its syntax.

It would only apply to headers explicitly extended to permit it. But we
are supposed to be writing a draft to deal with mailing lists, and that
would be a proper place to define such extensions (for use within the
UTF8SMTP universe).
Mailto: URIs for Chinese email addresses will be entirely
incomprehensible, either to a Chinaman or to anyone else, and yet people
will be expected to click on them (e.g. to unsubscribe from the list).
This is precisely the sort of situation which, according to the framework
document, we are trying to avoid, both within MUAs and, wherever possible,
on the wire also.

> But that idea was already rejected by the WG.

When and where? Can you point me to a message on this list which states
that?

>  More important,
> case-insensitive.

I don't know how you can say that. Punycode is defined in RFC 3492, which
states that case-sensitivity is REQUIRED for the basic Punycode algorithm.
One of the authors of that document has explicitly confirmed that on this
list.

I will grant you that it is possible to write a wrapper around punycode
which will arrange for case folding, and the model implementation provided
in RFC 3492 provides an option which effectively does just that (and it
may be a sensible option to use when coding domain-names). Naturally, we
would need to point out, in a NOTE, that that option should not be used
when encoding local-parts.

---

From: Kari Hurtta <[email-masked]>
Date: 23 Oct 2006 20:36:01 +0300

[email-masked] writes in gmane.ietf.ima:

  [ probably "Charles Lindsey" <[email-masked]>: ]
> methods for each headers.

Yes.

But I think that it make sense to require conversion only for some
subset of header fields.

Other should be optional (ie. just encapsulate them to Downgraded: -header
field, if corresponding downgrading method is not implemented. )

Minimal implementation should not be required to know long list
of different downgrading methods.

(IMHO for minimal implementation  RFC 2047 + RFC 2231 should be enough. )

> Some headers like 'Subject' are simply encoded by RFC 2047.

These header fields are
   
        from RFC 2822:

subject         =       "Subject:" unstructured CRLF

comments        =       "Comments:" unstructured CRLF

optional-field  =       field-name ":" unstructured CRLF

        from RFC 2045:

    description := "Content-Description" ":" *text

/ Kari Hurtta

---

From: Bill McQuillan <[email-masked]>
Date: Mon, 23 Oct 2006 10:50:52 -0700

> currently allows only headers starting with "Content-" there).

A small point: MIME body part headers MAY have header fields other than
Content-*.

From RFC 2046, Sec. 5.1. para 3:
   The only header fields that have defined meaning for body parts are
   those the names of which begin with "Content-".  All other header
   fields may be ignored in body parts.  Although they should generally
   be retained if at all possible, they may be discarded by gateways if
   necessary.  Such other fields are permitted to appear in body parts
   but must not be depended on.  "X-" fields may be created for
   experimental or private purposes, with the recognition that the
   information they contain may be lost at some gateways.

-- 
Bill McQuillan <[email-masked]>

---

From: John C Klensin <[email-masked]>
Date: Mon, 23 Oct 2006 20:34:57 -0400

--On Monday, 23 October, 2006 22:40 +0100 Charles Lindsey
<[email-masked]> wrote:

> On Mon, 23 Oct 2006 16:32:56 +0100, John C Klensin
> <[email-masked]> wrote:
> 
> Since John's reply didn't go to the whole list (my fault
> because he wasreplying to a copy addressed only to him), I am
> quoting the Whole of hisreply here.

I caught it this time, and am replying to the list as well.
But, out of curiosity, is there some reason why your MUA and/or
MTA cannot handle a message to someone that contains a "cc" to a
list, in violation of the clear intent of 821/822/2821/2822? 

Material below on which we agree snipped out...

>> I believe that is a correct reading.  But you and Frank are
>> doing a fairly good job of convincing me that either it, or
>> the text in ...eai-downgrade-... about handing those headers,
>> is wrong -- not the reading, but the text.
> 
> I certainly agree that there is a problem to be fixed one
> place or theother.

>...
>>>> This is _not_ a proposal, and I hope we don't need to go in
>>>> this direction.  However, it occurs to me that one way out
>>>> of many of these problems would be to require that the host
>>>> proposing to downgrade a message check each header for the
>>>> presence of 8bit characters.
>>> 
>>> I think that if a downgrader sees a Header-Type: UTF8SMTP at
>>> the toplevel, and if the C-T is some kind of multipart, then
>>> it has to decend(recursively if necessary) through the
>>> multipart structure looking forContent-* headers with
>>> <comment>s or <parameter>s with UTF-8 in them. Andat the same
>>> time it needs to look at the headers of any message/rfc822
>>> itencounters (though checking for the presence/absence of
>>> Header-Type:UTF8SMTP in there should suffice). You might even
>>> insist on a Header-Typeheader in the introductory heading of
>>> such parts (though RFC 2045/6currently allows only headers
>>> starting with "Content-" there).
>> 
>> In the general case, this requires looking through things it
>> doesn't understand.  For example, would a future
>> "message/foobar" be equivalent to "message/rfc822" wrt
>> interpretation of something that _might_ be a header or might
>> be an example in running text?
> 
> That would depend on the document that defined message/foobar.
> Ideally,that will happen well into the future, by which time
> UTF8SMTP will befamiliar to everyone and the guy writing that
> document would know he hadto address the issue. Alternatively,
> one might say that inventing a newmessage type and getting it
> widely accepted involves so many hoops thatnobody is ever
> going to do it.

My prediction is that we are going to need either a new message/
subtype, some retroactive changes to the definitions of existing
types, or both, in order to support a clean and reversible
encapsulation for forwarding purposes.  That implies "really
soon", even if we conclude it isn't on the critical path for
this WG.  YMMD, of course.
 
>>>>> Content-Disposition contains a filename parameter in UTF-8.
>>>>> RFC 2231 is the proper downgradethere (ugly, but it works).
>>>> 
>>>> Actually, it doesn't work.  In the general case, the
>>>> downgrading MTA can't possibly know anything about the
>>>> syntax of a filename on the system containing the delivery
>>>> MUA (not just MTA) -- that filename is interpreted on the
>>>> far side of POP or IMAP servers if they are used.
>>> 
>>> There is no requirement in RFC 2183 for the sending MUA or
>>> anyintermediate MTA to avoid filnames which might not be
>>> syntacticallyacceptable at the receiving end. The 'filename'
>>> parameter is just a'suggestion', and it clearly states that
>>> it is up to the receiving MUA todecide whether it is fit to
>>> be used as it stands.
>> 
>> Yes.  That might be an argument for dropping the filename
>> parameter if one downgrades... the principle should be, IMO,
>> that a false positive is much worse than a failure.
> 
> I don't see why. If it has been downgraded using RFC 2231,
> then maybe evenexisting implementations can cope with it. If
> not, then they just decidethat it is gibberish and move on to
> Plan B (which would be to invent anduse an arbitrary name).

The difficulty is that an existing implementation gets to look
at a string that might be some encoded words and decide whether
to interpret it as (i) a literal filename, (ii) a filename in
2231 encoding, (iii) something else.  It may expect ASCII-ified
non-ASCII filenames to be represented in HTML-like, or URI-like
offset notation, not as encoded words (indeed, that
interpretation might be fairly logical).    If it is a legacy
implementation that doesn't know about these things, the text of
2045 and 2047 --if I remember them correctly-- strongly
discourage interpretation (ii) by prohibiting the use of encoded
words in filenames.  While 2231 mentions filename parameters in
its introduction, it does not explicitly permit encoded words as
the value of that parameter.  More important, even if it did,
there is ample reason to believe that that there are legacy
implementations around that support 2045 and 2047 but not 2231
and 2183.   So the conservative strategy for the downgrading
system is to remove any filename parameters that contain
non-ASCII characters because it is the only want that it can
know that it isn't passing along trash as advice.   Again, you
may disagree, but that is "why".

>>> Moreover, RFC 2183 clearly states that, although filenames
>>> would beexpected to be in ASCII at the time of writing, it
>>> was to be expected thatfuture enhancements would make
>>> provision for Non-ASCII filenames. It mightbe argued that
>>> RFC 2231 had already provided that possibility, but if
>>> not,then it is clearly the case that our present UTF8SMTP
>>> has now providedprecisely the opportunity envisaged. So it
>>> is not a question of _whether_we permit ASCII filenames, but
>>> _how_. And a downgrade via RFC 2231 is theobvious mechanism
>>> (though we might encourage some normalization, probablyNFKC,
>>> while we are about it). And whatever provision we make
>>> should applyto <parameter>s in general. Naturally, <token>s
>>> will remain in ASCII, aswell as values that are keywords
>>> (such as the IANA-registered charsets).Non-ASCII boundaries,
>>> anyone?

>> I really see no problems with allowing non-ASCII filenames.
>> My problem is only with how to downgrade them.   And you can't
>> normalize in-transit unless you know that the target system
>> normalizes its file names (and how -- remember that there are
>> many situations in which NFD is more useful, and certainly
>> more consistent and predictable, than NFC), which you can't
>> possibly know.
>> 

Again, that this is written does not make it safe or indicate
that all cases have been evaluated.  I think this discussion is
demonstrating that the latter has not, in fact, been done.  The
key question, then, IMO, is whether making some retroactive
changes and doing a bit of hand waving is worth the risk of
botching something rather seriously.  You presumably feel that
is fairly safe.  I actually agree with you -- if there are file
systems around whose names look like encoded words, I certainly
haven't see them.    But my understanding of our obligation to
the installed email base is that we are obligated to introduce
_no_ new risks if there is any possible way to avoid them. 

In the examples you give above, the headers are reasonably
well-specified, although I'm still concerned about what to do
about ones that are not well-known and well-supported in legacy
implementations.  But it gives me no idea what to do about the
arbitrary "foo:" header.
 
>>>> But even knowing that something is an IRI or URI requires
>>>> either heuristics (bad news) of knowledge of a particular
>>>> header and its syntax.
>>> 
>>> It would only apply to headers explicitly extended to permit
>>> it. But weare supposed to be writing a draft to deal with
>>> mailing lists, and thatwould be a proper place to define such
>>> extensions (for use within theUTF8SMTP universe).
>> 
>> Very possibly.  But that isn't a comment on downgrade, but on
>> the mailing list issue.
> 
> Yes, but whenever we change one of the documents in our set,
> then weusually have to change the downgrade document to match.
> But it really istime that someone embarked on the mailing-list
> draft which we arechartered to write.

See draft-ietf-eai-mailinglist-00.txt.  It may or may not be
right, but the task has certainly been embarked upon.
 
>>> Mailto: URIs for Chinese email addresses will be
>>> entirely incomprehensible, either to a Chinaman or to anyone
>>> else, and yet peoplewill be expected to click on them (e.g.
>>> to unsubscribe from the list).This is precisely the sort of
>>> situation which, according to the frameworkdocument, we are
>>> trying to avoid, both within MUAs and, wherever possible,on
>>> the wire also.
>> 
>> I don't understand your second sentence above.
> 
> It was an allusion to the passage, in 4.2 of your framework
> draft,
> 
>     ... If non-obvious encodings,
>     such as protocol-specific ASCII-Compatible Encoding (ACE)
> variants,
>     are used, the user will inevitably, if only occasionally,
> see them
>     rather than "native" characters and will find that
> discomfiting or
>     astonishing.  Similarly, if different codings are used for
> mail
>     transport and message bodies, the user is particularly
> likely to be
>     surprised, if only as a consequence of the
> long-established "things
>     leak" principle.  The only practical way to avoid these
> sources of
>     discomfort, in both the medium and the longer term, is to
> have the
>     encodings used in transport be as nearly as possible the
> same as the
>     encodings used in message headers and message bodies.
> 
> Note that, from the POV of a Chinese email address, a URI is
> just another"ACE".

Hmm.  I don't think so.  But I'm not a Chinese speaker or user
of systems whose primary locale is based on Chinese.

>...
>> One could do that, but one would then have to figure out
>> whether any of the normalization, etc., issues impacted
>> whatever you then mean by a punycode representation.   But
>> then you would be introducing problems with the law of least
>> astonishment: Use of the same xn-- prefix when names were
>> going to be interpreted in a case-sensitive way that was used
>> for the DNS-like interpretation would be bad news.  In
>> addition, please remember that xn--gibberish is a value
>> local-part, or part of a local-part, without any decoding and
>> that, unlike the DNS, one can't sweep an appreciable fraction
>> of the local parts in use to know that it doesn't exist.
>> And, as I tried to point out in my note to Frank, if the
>> local part consists of logical components, rather than being
>> what we called "atomic" for a while, one has to choose
>> between dropping the whole thing into punycode (thereby
>> potentially hiding important information) and trying to
>> individually code its components (thereby introducing some
>> rather difficult lexical/ parsing issues).
> 
> Normalizations and the addition of the xn-- prefix are nothing
> to do withPunycode, and are not mentioned in RFC 3492. Any
> normalizations we imposedon local-parts would be decided by us
> (SASLPREP has been mentioned, Ibelieve), and naturally we
> would not use the xn-- prefix (I have suggestedthe prefix
> "yn--" for our case).
> 
> And even if, perchance, someone has used something beginning
> with yn-- ashis genuine local-part, that would be no different
> from someon whogenuinely intended to to use
>      =?iso-8859-1?Q?RE=3A_=5BSome=A0Nonsense?=
> as the Subject of his message. It is an acceptable risk.

See comments above about levels of acceptable risk (although I
don't expect you to agree).  "Subject" (and comments) are, IMO,
much less problematic that addresses and file names in that
regard since, other than possibly sorting messages after
[MTA]delivery, no automated actions are taken on them.

    john

---

From: Kari Hurtta <[email-masked]>
Date: 24 Oct 2006 07:26:56 +0300

> may disagree, but that is "why".

I'm unable to follow this.

   1)  If RFC 2231 encoding is done, it renames parameter names.
       Therefore there is not "filename" parameter for agents,
       which do not follow RFC 2231.

   2)  If there RFC 2047 encoded value on parameter in first
       place, looking for non-ASCII characters do not found it.

       Therefore  RFC 2047 garbage is passed anyway -- even
       when non-ASCII parameters are removed.

Can you specify on more detail on what sequence of events you are
thinking?

/ Kari Hurtta

---

From: Kari Hurtta <[email-masked]>
Date: 24 Oct 2006 12:11:21 +0300

> message must be rejected or an NDN issued.

Unknow headers can be encapsulated and original header can be 
then removed.

That was on my suggestion
 
   From: Kari Hurtta <[email-masked]>
   Subject: [EAI] draft-ietf-eai-utf8headers-01.txt /
     draft-ietf-eai-downgrade-02.txt suggestion
   Newsgroups: gmane.ietf.ima
   Cc: Kari Hurtta <[email-masked]>
   Date: Sat, 14 Oct 2006 14:48:07 +0300 (EEST)

draft-ietf-eai-utf8headers-01.txt part of that suggestion
seems to have realized. I do not know draft-ietf-eai-downgrade-02.txt
part of that suggestion yet.

/ Kari Hurtta

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 27 Oct 2006 22:55:21 +0100

> list, in violation of the clear intent of 821/822/2821/2822?

<rant>
I heartily dislike receiving personal copies of messages sent to a mailing  
list. Hence, I always deliver my replies to the List only. Unfortunately,  
MUAs rarely provide a "reply to list" capability, so you have to dig  
around to see whether the list itself is in the Cc or the To header (both  
are seen), and edit things around. If you hit 'R', then you have to edit  
out the unwanted ones. If you hit 'r', you have to manually edit in the  
list address instead of the original sender. The 2nd way is easier, but  
more error prone.

Periodically, people come up with perfect;y good and workable solutions to  
this problem on the ietf-822 mailing list. Mostly, these are well  
acccepted, but always the "old guard" on that list predict that what is  
proposed will inevitably lead to the premature Heat-Death of the Universe,  
and nothing ever comes of it.
<rant/>

> 2231 encoding, (iii) something else.

We are talking about filename parameters as defined in RFC 2183, although  
simiular considerations would apply to _any_ such <parameter> in _any_  
header where <paramater>s are allowed. Of the three cases you discuss, (i)  
and (ii) can be distinguished syntactically.

... filename = "foobar"
... filename*=utf-8'fr-FR'nom%20de%20l'%e9l%e8ve

case (iii) cannot arise (it would not be compliant with RFC 2183).

>  It may expect ASCII-ified
> interpretation might be fairly logical).

But not permitted by RFC 2183 (or alternatively, it is up to the operating  
system whether to understand filenames in some non-textual notation).
>    If it is a legacy
> words in filenames.

"encoded-words" (in the RFC 2047 sense) are entirely forbidden in  
<parameter>s, with or without RFC 2231.

>  ...  More important, even if it did,
> and 2183.

A legacy implementation that does not supports RFC 2183 will ignore  
Content-Distribution headers entirely. If it supports 2183 but not 2231,  
it will say "I see no 'filename' <parameter> here. I shall ignore the  
'filename*' <parameter> because I know of no such <parameter> in RFC  
2183". Either way, it is fail-safe.

>>> normalizes its file names

Any normalization that is done will be done by the originating MUA, or not  
at all.

> implementations.  But it gives me no idea what to do about the

The only thing you can do with an arbutrary "foo:" header is to replace it  
with

    Downgraded: foo: <encoded-word>

The target legacy implementation will either see it as a "Downgraded:"  
header or as a "foo:" header. In either case, it is supposed to ignore it,  
except perhaps for displaying it to the recipient.

>>>>> either heuristics (bad news) of knowledge of a particular

It requires knowledge of the particular header (hence my proposal  
explicitly to permit IRIs in those List-* headers).

> right, but the task has certainly been embarked upon.

Ah! I must have missed that one. Thanks.

> of systems whose primary locale is based on Chinese.

The only way to indicate 8-bit stuff in a URI is in hexadecimal, with each  
octet preceded by a '%'. That is at least 6 ASCII characters per UTF-8  
character (and probably more way up in the Chinese parts of Unicode). That  
is an ACE by any reckoning, and a singularly ugly one at that.

As an IRI, it would look like Chinese.

> [MTA]delivery, no automated actions are taken on them.

If we allow automatic downgrading of local-parts AT ALL, whatever ACE we  
use will have to produce a syntactically correct RFC 2822 local-part.  
Since that RFC 2822 local-part can always (however improbably) be a  
genuine ASCII local-part for some user in that domain, we can do nothing  
about it except ensure it is exceedingly improbable.

Choosing any other ACE than punycode does not change the problem in any  
way. A "magic prefix" will still be required. I would suggest that the  
prefex "yn--" is as improbable as anything else one might come up with.

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 28 Oct 2006 15:45:32 +0100

On Mon, 23 Oct 2006 18:50:52 +0100, Bill McQuillan <[email-masked]>  
wrote:

>> ... You might even insist on a Header-Type

OK, I had mis-remembered the precise restriction.
>    information they contain may be lost at some gateways.

However, that seems to say that any header MAY appear as MIME body part  
header, but if it does not begin with Content-* then it may be ignored,  
and even discarded en route. Which, in effect, is much the same as what I  
said. So tying to introduce an extension for using a Header-Type header in  
there would be somewhat dangerous.

---

From: "Yao Jiankang" <[email-masked]>
Date: Tue, 31 Oct 2006 16:11:01 +0800

----- Original Message ----- 
From: "Frank Ellermann" <[email-masked]>
To: <[email-masked]>
Sent: Saturday, October 21, 2006 2:11 PM
Subject: Re: [EAI] Comments on draft-ietf-eai-downgrade-02

> outcome of "elsewhere is irrelevant" (?)

In last IETF Montreal meeting, the WG has gotten a rough consensus about removing the atomic parameters.
We may not fully reject the idea of "translating the local part to ACE". It is up to users who decides whether or not "translating the local part to ACE" as the alt-address. but we suggest  not "translating the local part to ACE" to prevent the future possible potential problems caused by ACE.

Yao Jiankang
CNNIC

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Oct 2006 13:11:22 -0000

> removing the atomic parameters.

Yes, but not all the WG was present, or able to be present, at Montreal.  
And no announcement was made to this list that such a consensus had been  
reached.

Since then, that matter has been raised again and several people seem to  
wish to pursue the possibility further.

It has now been established that punycode is NOT case-insensitive  
(provided you do not confuse "pure punycode" from RFC 3492 with the  
additional cruft contained in "IDNA" from RFC 3490). That earlier  
misunderstanding may have influenced the discussions at Montreal.

It seems now that the only technical objection to punycode is that it may  
downgrade into an RFC 2822 <local-part> that is already the name of some  
other user in the relevant domain. But if the downgraded <local-part> is  
prefixed by something like "yn--", that is pretty unlikely, and a  
well-organized domain that had both UTF8 and ASCII users could easily  
ensure that this situation did not arise.

Moreover, this problem would arise with any ACE that might be proposed. So  
if punycode is to be rejected for this reason, then any other ACE must  
also be rejected.

I think we earlier agreed that if some "ATOMIC" parameter was to be  
associated with any such downgrade, then the keyword to be used should be  
"DOWNGRADABLE".

I therefore request the Chairs to consider whether, in view of these  
recent discussions, the possibility of using some automatic downgrade of  
<local-part>s can be put back on our Agenda.

(And punycode, if its usage is carefully worded, still seems the best  
candidate for the purpose.)

---

From: Kari Hurtta <[email-masked]>
Date: 31 Oct 2006 16:13:30 +0200

"Charles Lindsey" <[email-masked]> writes:
<...>
> associated with any such downgrade, then the keyword to be used should
> be  "DOWNGRADABLE".

Side note:

   I do not understand why keyword is needed (on place of alt-address).

   If origin host / mail domain is arranged automatic upgrading to 
   some punycode form, it can directly put that downgraded form to 
   alt-address.

> recent discussions, the possibility of using some automatic downgrade
> of  <local-part>s can be put back on our Agenda.

  Some informational RFC may be usefull for "Automatic generation of
  downgraded addresses from UTF8 address" which gives recommentation
  for origin host how that to do.

  ( Origin host have also full possibility to preserver all possible
    special characters (such as '+') and special prefixes and suffixes
    on original places, when doing downgrading, if it wants do it that
    way. )

/ Kari Hurtta

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Oct 2006 13:58:54 -0500

--On Tuesday, 31 October, 2006 13:11 +0000 Charles Lindsey
<[email-masked]> wrote:

>>> When was that ?  I vaguely recall a statement posted by
>>> Harald, and a debate about consensus on the list or
>>> elsewhere, with an outcome of "elsewhere is irrelevant" (?)
>> 
>> 
>> In last IETF Montreal meeting, the WG has gotten a rough
>> consensus about   removing the atomic parameters.
> 
> Yes, but not all the WG was present, or able to be present, at
> Montreal.And no announcement was made to this list that such a
> consensus had beenreached.
> 
> Since then, that matter has been raised again and several
> people seem towish to pursue the possibility further.
> 
> It has now been established that punycode is NOT
> case-insensitive(provided you do not confuse "pure punycode"
> from RFC 3492 with theadditional cruft contained in "IDNA"
> from RFC 3490). That earliermisunderstanding may have
> influenced the discussions at Montreal.
> 
> It seems now that the only technical objection to punycode is
> that it maydowngrade into an RFC 2822 <local-part> that is
> already the name of someother user in the relevant domain. But
> if the downgraded <local-part> isprefixed by something like
> "yn--", that is pretty unlikely, and awell-organized domain
> that had both UTF8 and ASCII users could easilyensure that
> this situation did not arise.

To put part of that objection slightly more precisely, there is
a concern that a delivery SMTP server system might be using a
coding convention for the local part that selected names with
particular character sequences for special processing.  For
example, it would not be terribly unreasonable for a delivery
system to have a rule that any string that contained two
consecutive hyphens was to be routed to one system or set of
systems for processing as commands while anything without that
string was to be routed to a different system or subsystem for
delivery (or processing) as mail.  Depending on the order in
which processing occurred and a few other issues, users sending
to a "yn--" downgraded address might get rather confusing error
messages... a problem that could not be avoided simply by the
domain's choice of addresses to be used, especially if it had
not been upgraded to deal with i18n email addresses, downgraded
or not.

The other part of that objection is the comment in "framework"
about having different encodings for the local part and the
domain part.  That is a preference, not a firm requirement, but
the reasons for the preference in terms of avoiding user
confusion or astonishment should be fairly clear.

> Moreover, this problem would arise with any ACE that might be
> proposed. Soif punycode is to be rejected for this reason,
> then any other ACE mustalso be rejected.

That is, indeed, the reason why much of the discussion --and the
language in "framework" and its predecessors-- focuses on
explicit, destination-specified, alternate addresses (or
destination-authorized use of a particular ACE via the late
ATOMIC option), rather than arbitrary downgrading to a
sender-selected, or standard-selected, ACE.

> I think we earlier agreed that if some "ATOMIC" parameter was
> to beassociated with any such downgrade, then the keyword to
> be used should be"DOWNGRADABLE".

My impression continues to be that we gave up on that whole
model, with a preference to instead simply specify the alternate
address.  I'll leave the question of whether there was, or was
not, consensus on that point in the hands of the co-chairs.

> I therefore request the Chairs to consider whether, in view of
> theserecent discussions, the possibility of using some
> automatic downgrade of<local-part>s can be put back on our
> Agenda.
> 
> (And punycode, if its usage is carefully worded, still seems
> the bestcandidate for the purpose.)

This is certainly a request you can make.  I hope they will
respond in a timely fashion.

       john