Subject: [EAI] ATOMIC or not

From: "Edmon Chung" <[email-masked]>
Date: Tue, 11 Jul 2006 13:54:08 -0400

During the session today in Montreal, a discussion was raised about the use of the ATOMIC parameter.  This was brought up during the discussion of the consolidation of using one instead of 2 parameters in the SMTP extension, as well as the syntax of whether such a parameter should be within the regular address angled brackets "<>" or outside.  A suggestion was made to eliminate the use of the ATOMIC identification completely and a consensus was sought, which showed support in the room.  However, it occurred to me that many in the room was a bit confused about what was being decided.  This is exemplified I think by the continued discussion about having an ATOMIC "transformed" format of an internationalized email address in the subsequent draft discussions by the respective authors.

Because almost all of the current drafts include the concept of having an ATOMIC parameter, I think it is prudent that we have a more indepth discussion on the subject.

Before jumping to the discussion, it is useful I think to point out that (at least as far as my understanding goes) the use of ATOMIC presumes that in conjunction there is a generally known algorithm that can be used to transform ("downgrade" or "upgrade") an internationalized email address.  Therefore, if an internationalized email address can be such "transformed", then the ATOMIC bit can be set.  This will allow mail servers to downgrade/upgrade an address when needed.

From both the Headers document as well as the Downgrade document, it seems to me that the ATOMIC "flag" may be useful.  The presentation from Nai Wen (TWNIC), especially his trial/demo discussion, also seem to suggest that having the ATOMIC (more importantly the algorithm for transforming to a downgraded format) functionality maybe very useful.

The argument put forth against having ATOMIC was clear: the sender could always use a particular algorithm to provide an alt-address and therefore, simply having an option to announce an alt-address is good enough.

The argument for having the ATOMIC was not that clear (perhaps others can add).  In my mind, I think the usefulness of having the ATOMIC flag is not only that it allows a more concise way of sending/storing downgrade data, it is much better for transportability and passing around.  Because mail is passed around siginificantly, this is an important consideration in my mind.  What had NOT been discussed in the current set of drafts is whether an "alt-address" can survive more than one series of transport.  I.e., if I have obtained an alt-address from a particular internationalized email address before, can I use it again in the future?... With ATOMIC, this can be safely assumed.  With alt-address, this cannot be assumed.  Without the ability of using downgrade information, downgrade may in itself become much less useful, because when downgrade is most needed, often the downgrade information would not be available.  Even if I have obtained an alt-address before I cannot assume that I can use it again (or can I?)... but with ATOMIC, I can assume that the downgrade is applicable in the future.  An MUA or an MTA could therefore store these information for meaningful reuse.

The problem overall of course is that (if my assumption earlier was correct), and if we keep ATOMIC, we will need to define a known algorithm for transforming an address.

Anyway, an important point is, at least for myself, I am not ready to completely eliminate ATOMIC yet until there is further thought on the issues at stake.  I suspect there are others that share my concern... and more that are just generally confused because of the number of inconsistencies between the current set of drafts... Perhaps if and when the drafts are more consistent, the usefulness (or uselessness) of the ATOMIC parameter/option can be better evaluated...

Edmon

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Nai-Wen Hsu <[email-masked]>
Date: Wed, 12 Jul 2006 02:40:28 +0800

Although the trial system very depend on the ATOMIC flag to transform 
the utf8 email address to downgraded format, but I think it is ok to 
remove ATOMIC flag, because MUA can transform to downgraded fromat and 
put it into alt-address before sending out the mail.

Nai-Wen Hsu

Edmon Chung :

>------------------------------------------------------------------------
>

---

From: Yangwoo Ko <[email-masked]>
Date: Wed, 12 Jul 2006 03:46:27 +0900

Edmon Chung wrote:
> During the session today in Montreal, a discussion was raised about
> the use of the ATOMIC parameter.  This was brought up during the
> discussion of the consolidation of using one instead of 2 parameters
> in the SMTP extension, as well as the syntax of whether such a
> parameter should be within the regular address angled brackets "<>" or
> outside.  A suggestion was made to eliminate the use of the ATOMIC
> identification completely and a consensus was sought, which showed
> support in the room.  However, it occurred to me that many in the room
> was a bit confused about what was being decided.  This is exemplified
> I think by the continued discussion about having an ATOMIC
> "transformed" format of an internationalized email address in the
> subsequent draft discussions by the respective authors.
> 
> Because almost all of the current drafts include the concept of having
> an ATOMIC parameter, I think it is prudent that we have a more indepth
> discussion on the subject.
> 
> Before jumping to the discussion, it is useful I think to point out
> that (at least as far as my understanding goes) the use of ATOMIC
> presumes that in conjunction there is a generally known algorithm that
> can be used to transform ("downgrade" or "upgrade") an
> internationalized email address.  Therefore, if an internationalized
> email address can be such "transformed", then the ATOMIC bit can be
> set.  This will allow mail servers to downgrade/upgrade an address
> when needed.
> 
> From both the Headers document as well as the Downgrade document, it
> seems to me that the ATOMIC "flag" may be useful.  The presentation
> from Nai Wen (TWNIC), especially his trial/demo discussion, also seem
> to suggest that having the ATOMIC (more importantly the algorithm for
> transforming to a downgraded format) functionality maybe very useful.
> 
> The argument put forth against having ATOMIC was clear: the sender
> could always use a particular algorithm to provide an alt-address and
> therefore, simply having an option to announce an alt-address is good
> enough.
> 
> The argument for having the ATOMIC was not that clear (perhaps others
> can add).  In my mind, I think the usefulness of having the ATOMIC
> flag is not only that it allows a more concise way of sending/storing
> downgrade data, it is much better for transportability and passing
> around.  Because mail is passed around siginificantly, this is an
> important consideration in my mind.  What had NOT been discussed in
> the current set of drafts is whether an "alt-address" can survive more
> than one series of transport.  I.e., if I have obtained an alt-address
> from a particular internationalized email address before, can I use it
> again in the future?... With ATOMIC, this can be safely assumed.  With
> alt-address, this cannot be assumed.  Without the ability of using
> downgrade information, downgrade may in itself become much less
> useful, because when downgrade is most needed, often the downgrade
> information would not be available.  Even if I have obtained an
> alt-address before I cannot assume that I can use it again (or can
> I?)... but with ATOMIC, I can assume that the downgrade is applicable
> in the future.  An MUA or an MTA could therefore store these
> information for meaningful reuse.

We can provide an ALT-ADDR of an i18n email address because the holder
of the address gave that information. And this is same for ATOMIC.

Association of of an ALT-ADDR and an i18n address can be broken in the
future. And, once again, so is ATMOCity of an address. However the
latter is far less probably than the former.

> 
> The problem overall of course is that (if my assumption earlier was
> correct), and if we keep ATOMIC, we will need to define a known
> algorithm for transforming an address.

Even though we remove ATOMIC as an element of protocol specification,
we can still reuse the concept as a part of email address management
practice supported by MUAs.

> 
> Anyway, an important point is, at least for myself, I am not ready to
> completely eliminate ATOMIC yet until there is further thought on the
> issues at stake.  I suspect there are others that share my concern...
> and more that are just generally confused because of the number of
> inconsistencies between the current set of drafts... Perhaps if and
> when the drafts are more consistent, the usefulness (or uselessness)
> of the ATOMIC parameter/option can be better evaluated...
>
> 
> Edmon
> 
> 
> 
>

---

From: Randall Gellens <[email-masked]>
Date: Tue, 11 Jul 2006 13:19:06 -0700

At 1:54 PM -0400 7/11/06, Edmon Chung wrote:

> discussion of the consolidation of using one instead of 2 
> parameters in the SMTP extension, as well as the syntax of whether 
> such a parameter should be within the regular address angled 
> brackets "<>" or outside.  A suggestion was made to eliminate the 
> use of the ATOMIC identification completely and a consensus was 
> sought, which showed support in the room.  However, it occurred to 
> me that many in the room was a bit confused about what was being 
> decided.

I'd like to clarify the proposal was to eliminate ATOMIC by combining 
it with ALT-ADDRESS.  More details in-line.

>   This is exemplified I think by the continued discussion about 
> having an ATOMIC "transformed" format of an internationalized email 
> address in the subsequent draft discussions by the respective 
> authors.
>
>  Because almost all of the current drafts include the concept of 
> having an ATOMIC parameter, I think it is prudent that we have a 
> more indepth discussion on the subject.
>
> presumes that in conjunction there is a generally known algorithm 
> that can be used to transform ("downgrade" or "upgrade") an 
> internationalized email address.

The proposal does not alter the presence of this algorithm, it only 
alters who uses it and when in the chain.

>    Therefore, if an internationalized email address can be such 
> "transformed", then the ATOMIC bit can be set.  This will allow 
> mail servers to downgrade/upgrade an address when needed.

The proposal is that, rather than carry around the ATOMIC flag and no 
ALT-ADDRESS, just go ahead and use the algorithm on atomic addresses 
for which no other alternative address is available.

Here are the cases:
	1: client knows a good alternate address.

	2: client does not have a good alternate address, but it knows 
the address is transform-safe ("atomic")

	3: client does not have a good alternate address, and does not 
know if the address is transform-safe ("atomic")

So, here is the behavior:

	1: client knows a good alternate address.  It supplies it in ALT-ADDRESS.

	2: client does not have a good alternate address, but it knows 
the address is transform-safe ("atomic").  It performs the 
transformation and supplies this transformed address in ALT-ADDRESS.

	3: client does not have a good alternate address, and does not 
know if the address is transform-safe ("atomic").  It does not supply 
ALT-ADDRESS.

>
>>From both the Headers document as well as the Downgrade document, 
>> it seems to me that the ATOMIC "flag" may be useful.  The 
>> presentation from Nai Wen (TWNIC), especially his trial/demo 
>> discussion, also seem to suggest that having the ATOMIC (more 
>> importantly the algorithm for transforming to a downgraded format) 
>> functionality maybe very useful.

I agree that the more important aspect is the algorithm for doing the 
transformation (in a reversible way).  Passing around the ATOMIC flag 
seems much less useful, to me, as it only makes the SMTP chain more 
complex, without adding any extra benefit that I can see.

> could always use a particular algorithm to provide an alt-address 
> and therefore, simply having an option to announce an alt-address 
> is good enough.

The sender uses the same algorithm that ATOMIC would permit.  The 
question is simply where this algorithm is performed: at some 
arbitrary point in the SMTP chain, or only at the start?

>  The argument for having the ATOMIC was not that clear (perhaps 
> others can add).  In my mind, I think the usefulness of having the 
> ATOMIC flag is not only that it allows a more concise way of 
> sending/storing downgrade data, it is much better for 
> transportability and passing around.

More concise because it is shorter than ALT-ADDRESS?  Or more concise 
in some other way?

>   Because mail is passed around siginificantly, this is an important 
> consideration in my mind.

In what way?  For server storage of in-transport mail?

>    What had NOT been discussed in the current set of drafts is 
> whether an "alt-address" can survive more than one series of 
> transport.  I.e., if I have obtained an alt-address from a 
> particular internationalized email address before, can I use it 
> again in the future?... With ATOMIC, this can be safely assumed. 
> With alt-address, this cannot be assumed.

The proposal does nothing to change this -- it only discusses where 
the transformation is done in a particular message's path through 
SMTP.  It's the same transformation.

>    Without the ability of using downgrade information, downgrade may 
> in itself become much less useful, because when downgrade is most 
> needed, often the downgrade information would not be available. 
> Even if I have obtained an alt-address before I cannot assume that 
> I can use it again (or can I?)... but with ATOMIC, I can assume 
> that the downgrade is applicable in the future.  An MUA or an MTA 
> could therefore store these information for meaningful reuse.

This suggestion of re-using transformed addresses is not part of the proposal.

> algorithm for transforming an address.

Go ahead and define the algorithm, but we don't need the specific 
ATOMIC parameter to do so.

>
>  Anyway, an important point is, at least for myself, I am not ready 
> to completely eliminate ATOMIC yet until there is further thought 
> on the issues at stake.  I suspect there are others that share my 
> concern... and more that are just generally confused because of the 
> number of inconsistencies between the current set of drafts... 
> Perhaps if and when the drafts are more consistent, the usefulness 
> (or uselessness) of the ATOMIC parameter/option can be better 
> evaluated...

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
In politics stupididty is not a handicap.
      --Napoleon

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 11 Jul 2006 17:19:04 -0400

> know if the address is transform-safe ("atomic").  It does not supply 

I think this is clear.  The question perhaps from this is do we make the above a requirement? Or the client can always decide that it does not know even if it does?

> In what way?  For server storage of in-transport mail?

I can imagine a different way of storing the ATOMIC information so that it could be useful (e.g. for contact list kept by an MUA).

> This suggestion of re-using transformed addresses is not part of the proposal.

Yes and No.  If transformed addresses are NEVER "reused" then a recipient cannot reply using it... the scenarios describe the requirement for ability to reply.  So there you are still drawing a line... is it "reused" for the "same" message in its reply only?... can I use the transformed address again at some other time?  Another question following from that maybe... how long might an alt-address be "valid"?  Or should an MUA/MTA assume that an alt-address is always "valid"?

> ATOMIC parameter to do so.

There is no reason to have a general "standard" algorithm without the ATOMIC parameter.  Different transformation mechanisms may be used.

Edmon

---

From: Randall Gellens <[email-masked]>
Date: Tue, 11 Jul 2006 13:20:32 -0700

At 2:40 AM +0800 7/12/06, Nai-Wen Hsu wrote:

>   I think it is ok to remove ATOMIC flag, because MUA can transform 
> to downgraded fromat and put it into alt-address before sending out 
> the mail.

This is exactly the proposal.  Thank you for stating it so clearly.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Demagogue: One who preaches doctrines he knows to be untrue to men he
knows to be idiots.                                    --H.L. Mencken

---

From: Randall Gellens <[email-masked]>
Date: Tue, 11 Jul 2006 16:23:28 -0700

At 5:19 PM -0400 7/11/06, Edmon Chung wrote:

>  I think this is clear.  The question perhaps from this is do we 
> make the above a requirement? Or the client can always decide that 
> it does not know even if it does?

I'd suggest it is better to allow a client to act as if it doesn't 
know that an address is atomic, as the result (that the message is 
bounced if it can't be delivered as-is) may be the most desirable 
outcome in many cases.

>  I can imagine a different way of storing the ATOMIC information so 
> that it could be useful (e.g. for contact list kept by an MUA).

It seems to me that for any cases where storing ATOMIC would be 
useful, storing the transformed address would be equally as useful.

>  There is no reason to have a general "standard" algorithm without 
> the ATOMIC parameter.  Different transformation mechanisms may be 
> used.

I'm sorry, but I don't follow.  Can you explain more?  If different 
transformation algorithms are used, then how can the resulting 
transformed address be expected to succeed (be accepted by the 
recipient server and delivered to the same user as the UTF8 address)?

-------------- Randomly-selected tag: ---------------
The attention span of a computer is only as long as its electrical cord.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 12 Jul 2006 10:30:50 +0100

>> "transformed", then the ATOMIC bit can be set.  This will allow mail  
>> servers to downgrade/upgrade an address when needed.
>
> ALT-ADDRESS, just go ahead and use the algorithm on atomic addresses for  
> which no other alternative address is available.

That is not my understanding. It would be much simpler if all addresses  
were presumed to be "Atomic" *unless* an explicit Alt-Address was  
provided. But for that to work, we need an agreed downgrading algorithm  
that is perfectly reversible. In that case, if the downgraded address made  
no sense at the receiving end (and especially if it could be reliably  
detected as having been downgraded, e.g. by starting with 'xn--' or  
whatever), then the receiving end just reverses the downgrade, at which  
point the address should now make sense again (because it was as if there  
never had been any downgrade).

I do not like systems which expect the client to provide the ACEd address  
in the Alt-Address regardless. Most people using UTF8smtp will be  
communicating only with other people doing the same, and do not want to  
see unintelligible (to them) Alt-Addresses.
> 	2: client does not have a good alternate address, but it knows the  
> address is transform-safe ("atomic")

In my world, all addresses are 'transfer-safe' (because the ACE is always  
reversible). If that can be assumed, then all remaining problems go away.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Edmon Chung" <[email-masked]>
Date: Wed, 12 Jul 2006 16:38:35 -0400

> useful, storing the transformed address would be equally as useful.

Not true.  If an address is designated as ATOMIC, then I would store that information cause I know that it could be used in the future.  If an alt-address is provided, then I would probably have to discard it cause I cannot determine whether I can use that same alt-addr in the future.

> recipient server and delivered to the same user as the UTF8 address)?

Without ATOMIC, you cannot assume any ALT-ADDRESS can be transformed back to the UTF8 address.  If there is no way to designate that an address is ATOMIC, then one can only draw the relation between the UTF8 address and the ALT-ADDRESS for the particular transport at hand.

Edmon

---

From: "Edmon Chung" <[email-masked]>
Date: Wed, 12 Jul 2006 16:44:44 -0400

> were presumed to be "Atomic" *unless* an explicit Alt-Address was  

That is I think the confusion.  That is NOT the case based on the current specifications.  If an address is NOT designated as ATOMIC, it is assumed it CANNOT be transformed into any "ACE" address.

> whatever), then the receiving end just reverses the downgrade, at which  

Definitely not xn--, :-P...

> never had been any downgrade).

Again, as mentioned, that is not currently the case.  If an address is NOT designated as ATOMIC, we CANNOT downgrade (or for that matter upgrade) it.

> see unintelligible (to them) Alt-Addresses.

ALT-ADDRESS does NOT have to be unitelligible addresses (one reason I like alt-address), they could be just your traditional email address.  Whereas stating "ATOMIC" allows the transport to be cleaner as you mentioned, and to designate that it can be transformed into an ACE address.

> reversible). If that can be assumed, then all remaining problems go away.

That is not the assumption of the current set of drafts... if that is what we want, then we need to make some changes...

Edmon

---

From: Tony Finch <[email-masked]>
Date: Wed, 12 Jul 2006 22:20:13 +0100

On Wed, 12 Jul 2006, Edmon Chung wrote:
>
> If an address is designated as ATOMIC, then I would store that
> information cause I know that it could be used in the future.  If an
> alt-address is provided, then I would probably have to discard it cause
> I cannot determine whether I can use that same alt-addr in the future.

If you can't trust the alt-address then surely you can't trust the utf8
address either. Or in other words, I don't think this kind of operational
stability issue is relevant. You can always verify that the alt-address is
the algorithmically-downgraded form of the utf8 address by doing the
downgrade and comparing.

I'm seriously worried about the user-interface implications of non-atomic
addresses.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HEBRIDES BAILEY: WEST OR SOUTHWEST 5 TO 7, OCCASIONALLY GALE 8 AT FIRST,
DECREASING 3 OR 4. SHOWERS AT FIRST. MAINLY GOOD.

---

From: "Edmon Chung" <[email-masked]>
Date: Wed, 12 Jul 2006 19:04:03 -0400

----- Original Message ----- 
From: "Tony Finch" <[email-masked]>
> the algorithmically-downgraded form of the utf8 address by doing the

ALT-ADDRESS is not the same as ATOMIC.  ALT-ADDRESS can be any ASCII address.  E.g. "<chineseName>@domain.tld" can have an ALT-ADDR="[email-masked]" (note not even the domain needs to match).  That is why the alt-address cannot be a "stable" glue to the UTF8 address.  The UTF8 address can be fully trusted so can an ATOMIC address (well unless the standard transformation algorithm is changed... but that would be a different issue... more on versioning).

> I'm seriously worried about the user-interface implications of non-atomic

Well without the ATOMIC parameter at all there is no way to say that an address is atomic... and based on the current set of specifications, Non-atomic is the default.

Edmon

---

From: Yangwoo Ko <[email-masked]>
Date: Thu, 13 Jul 2006 09:01:37 +0900

Edmon Chung wrote:
> ALT-ADDRESS is not the same as ATOMIC.  ALT-ADDRESS can be any ASCII 
 > address.  E.g. "<chineseName>@domain.tld" can have
 > an ALT-ADDR="[email-masked]" (note not even the domain needs to match).
 > That is why the alt-address cannot be a "stable" glue to the UTF8
 > address.  The UTF8 address can be fully trusted so can an ATOMIC
 > address (well unless the standard transformation algorithm is
 > changed... but that would be a different issue... more on versioning).

No. As I argued in the previous mail, ATOMIC is just a little bit
more stable than ALT-ADDR. First of all, a sender may claim that a
given address is ATOMIC even though it is not. Second, association
of an ATOMIC address with its corresponding converted all-ASCII
address can never be enforced by protocols. This is just part of
administration that can be violated in so many ways.

---

From: Bill McQuillan <[email-masked]>
Date: Thu, 13 Jul 2006 00:24:28 -0700

I feel I need to delurk here just to point out what I am increasingly
seeing as a problem. I believe that several people on this list are
alluding to this in the latest exchange of messages.

The "alt-addr" mechanism seems to be becoming more and more an immense
administrative nightmare!

If I recall, it was proposed as a solution to the problem of "non-atomic"
addresses. (I hate that term!) That is, addresses that cannot be
transformed into an RFC2822-safe format for some reason. I do not feel that
this was explored extensively in the WG. (Admittedly, I was not at any
face-to-face meetings.)

If there existed a transformation that was guaranteed reversible and did
not apply to already RFC2822 compliant addresses or the ASCII portions of
the local parts, I have to agree with Charles Lindsey that many, many
problems would go away.

Note that any address that is currently parsed by the sender or recipient
must already be RFC2822 compliant and therfore would not be transformed.
Also, commonly known format of addresses like "-owner", "+folder", etc. use
acceptable ASCII chars ("-", "+") which should not be transformed by a
reasonable ACE and therfore should be parsed as expected. The only entity
that needs to know how to interpret the parsed portion of local parts is
presumably a final delivery agent that has been enhanced to accept UTF8
local parts and could easily be expected to be able to undo the ACE.

As a strawman, here is an ACE that follows these requirements (though
perhaps a little verbosely!):

If any character of a local part is not an allowed RFC2822-compliant
character, then the entire local part is preceded by a flag string such as
"ima--" and each such non-conpliant character is replaced in the local part
by "=XX" enough times to express each octet of the UTF8 encoding of the
character, where XX is the HEX expression of an octet. Any character that
*is* valid in an RFC2822 local part SHOULD NOT be transformed.

Let the brickbats fly!

-- 
Bill McQuillan <[email-masked]>

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 13 Jul 2006 16:46:43 +0100

> administrative nightmare!

It should not be. It should only be put there by the original author (or  
by some software configured with his approval - but remember that it can  
also turn up in a Reply-To address). I says "If this message (or replies  
to it) cannot be delivered to the UTF-8smtp address,l then I am happy for  
them to be delivered to this ascii@ascii address instead". Don't read any  
mor into it than that. Maybe the alt-address goes to the same destination.  
Maybe it goes to some agency which will translate the message and send it  
on. Maybe it goes to dev-null. You don't know - all you do know is that is  
what the original author had instructed.

> the local parts, I have to agree with Charles Lindsey that many, many

I believe it is possible to make an ACE that is guaranteed reversible.  
Since this latest thread started, I have been looking into Punycode again.  
But it is indeed full of worms, and I want to be sure I really understand  
them all before I report on them. But indications are that it can be made  
to work.

---

From: "Edmon Chung" <[email-masked]>
Date: Thu, 13 Jul 2006 12:31:01 -0400

----- Original Message ----- 
From: "Charles Lindsey" <[email-masked]>
> them all before I report on them. But indications are that it can be made  

One thing that IDN had to do was to try its best to avoid "conflicting" with existing names against the "converted form".  "XN--" prefix was finally chosen, and we have so far not heard any issue from previously existing domain that started with "xn--" that is now affected.

For mail username it might be much harder as there exists many forms of "encoded" usernames along with much more "usernames" in general, especially when the "converted form" must also be compliant with existing rules.  Not saying it is impossible, just need to have that in mind if we decide at all to go down the path of designing something for it.

The questions still remains (at least for this thread/exercise):
1. Do we want a parameter for designating ATOMIC or not?
2. If not, is ATOMIC the default? or Non-ATOMIC the default?
3. If non-ATOMIC is the default, there is no point creating the "guaranteed reversibe ACE"... is there?  Reason being that given a UTF8 address and that it is non-ATOMIC, we cannot then try to transform it into ACE for downgrade and expect it to work anyway.
4. OR is it that if a "guaranteed reversible ACE is defined, non-ATOMIC itself is no-longer "valid", i.e. they are mutually exclusive (non-atomic vs. guaranteed reversible ACE)?
5. At which point... then is alt-address needed at all?  Should it be out of scope and left for other extensions to provide an alias of an email address (regardless of EAI or not, for any address field)?... this question maybe going to far... but nevertheless :-P  (personally I like having the flexibility of having an alt-addr in this context though... that doesnt mean it is "necessary")

Edmon

---

From: Yangwoo Ko <[email-masked]>
Date: Fri, 14 Jul 2006 02:28:59 +0900

> 2. If not, is ATOMIC the default? or Non-ATOMIC the default?

Let's ask the same question to email addresses used today?
Which one is default? Well, my answer is none. Except the
final delivery, we do not assume anything about the nature
of local part. I want that the same thing is true for i18n
email addresses.

However, if we want to attach such information, we may choose
either ATOMIC or non-ATOMIC as default. When introducing
ATOMIC, we assumed that non-ATOMIC is default.

> 3. If non-ATOMIC is the default, there is no point creating the "guaranteed reversibe ACE"... is there?  

I cannot follow your logic here.

 >Reason being that given a UTF8 address and that it is non-ATOMIC, we 
cannot then try to transform it into ACE for downgrade and expect it to 
work anyway.

Yes.

> 4. OR is it that if a "guaranteed reversible ACE is defined, non-ATOMIC itself is no-longer "valid", i.e. they are mutually exclusive (non-atomic vs. guaranteed reversible ACE)?

This is related to when such a conversion is made. For
example, as is done in IDNA if it is always clear when
to convert (and revert back) and we use only the
converted on-the-wire, then maybe we don't care whether
given addresses are ATOMIC or not. But unfortunately,
this is not the case.

> 5. At which point... then is alt-address needed at all?  Should it be out of scope and left for other extensions to provide an alias of an email address (regardless of EAI or not, for any address f…

ALT-ADDR will be a lot more useful when we want to
send mails though mailing list. For one-to-one (or
one-to-few) communications, sender/recipients are
controlled by sender explicitly, the sender can be
wise enough to choose i18n addresses or all-ASCII
addresses so that the mail can be delivered as
wanted. In addition, even though there are error
in delivery, the sender can react upon them. (E.g.
sending the same mail with all-ASCII addresses.)
ALT-ADDR can be helpful to reduce the number of
these (probably manual) reactions.

On the other hand, senders are usually helpless
who are there on a given mailing list.

---

From: Tony Hansen <[email-masked]>
Date: Thu, 13 Jul 2006 16:18:13 -0400

Let's separate out the current ATOMIC implementation and the ATOMIC
concept. The ATOMIC concept says that the i18n localpart can be safely
downgraded to the downgradable ACE form. That's it.

During the EAI meeting, when people said to get rid of ATOMIC in SMTP,
my understanding is that they were saying to get rid of the SMTP ATOMIC
parameter. They were *not* saying anything about getting rid of the
ATOMIC concept.

If we want to support the ATOMIC concept, no matter how we do it, the
sender needs to know if any given i18n localpart is ATOMIC. This doesn't
necessarily mean that all intermediate hops in message transmission also
needs to know it.

Orthogonal to this concept is that of an alternate ascii-compatible
address that would be used if the i18n localpart cannot be used.

The ATOMIC information provides a way to indicate that the downgradable
ACE form of the i18n address can be used as the alternate address.

The single bit of knowledge about whether an i18n localpart is ATOMIC
can be used and passed in several ways. The current specs call for the
MUA to do the following processing:

mua:	if i18n address is ATOMIC
mua:		pass i18n address and info saying address is ATOMIC
mua:	else if there is an alternate ACE address
mua:		pass i18n address and alternate ACE address
mua:	else
mua:		pass i18n address

When an MTA hits another MTA that is not i18n compatible, it must do the
following processing:

mta:	if address is ATOMIC
mta:		downgrade message headers and body
mta:		convert i18n address to ACE
mta:		use converted address
mta:	else if there is an alternate ACE address
mta:		downgrade message headers and body
mta:		use alternate address
mta:	else
mta:		bounce the message

Consider the following alternative where the MUA uses the knowledge that
the i18n address is ATOMIC and uses that knowledge up front to set the
alternate address.

mua:	if i18n address is ATOMIC
mua:		pass i18n address and generated ACE address as alternate
mua:	else if there is an alternate ACE address
mua:		pass i18n address and alternate ACE address
mua:	else
mua:		pass i18n address

Now look at how much simpler the MTA becomes:

mta:	if there is an alternate ACE address
mta:		downgrade message headers and body
mta:		use alternate address
mta:	else
mta:		bounce the message		

During the EAI meeting, when people said to get rid of ATOMIC in SMTP,
my understanding is that they were saying to get rid of the SMTP ATOMIC
parameter, but not the ATOMIC concept.

	Tony Hansen
	[email-masked]

---

From: Edmon Chung <[email-masked]>
Date: Fri, 14 Jul 2006 22:32:58 -0400

I understand your analysis and the approach suggested.  Not saying it does not work...

However, my point is if "ATOMIC" as a parameter is never transported, then there is no need for a standardized way to create an ACE representation of a UT8 address.  Because each MUA can use any method of transformation to an alt-address they want and the protocol would still function without any problem.  Because in no place would the infrastructure attempt to up/down-convert the address.

Also, it will mean that there is no way i can obtain a relatively stable ASCII-representation/equivalent of an address.  In that case, when I re-initiate an email to a UTF8 address I would only be able to use the UTF8 address and cannot assume that the alt-address is still valid (unless we specify that in the standards).  This creates additional issue for situations of 3-party communications (scenario 2.5 in scenarios doc: http://www.ietf.org/internet-drafts/draft-ietf-eai-scenarios-01.txt).

To illustrate my point:

Lets say there are 3 parties: EAIuser1, EAIuser2 and ASCIIuserA

IF an ATOMIC parameter can be passed allong:

Lets say EAIuser1 sends to EAIuser2 and the path is fully EAI-aware, and that the ATOMIC parameter is passed along.  EAIuser2 can now store the information

THEN, EAIuser2 after a few weeks is interested to send email to EAIuser1 and ASCIIuserA together.  The transport EAIuser2->EAIuser1 can continue to use UTF8SMTP, and for EAIuser2->ASCIIuserA, a downgrade can happen by converting to ACE address based on the ATOMIC information.  Mail can be sent successfully.

IF however ATOMIC parameter cannot be passed along:

When EAIuser1 sends to EAIuser2, even if it provided the ALT-ADDRESS (no matter whether it was converted using an algorithm, because without the ATOMIC information we cannot assume it is)

WHEN EAIuser after a few weeks tries to initiate an email to EAIuser1 and ASCIIuserA together, it will not be able to successfully send to ASCIIuserA because the header downgrade could not happen for the field containing EAIuser1's address.

Edmon

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 15 Jul 2006 20:42:15 +0100

On Sat, 15 Jul 2006 03:32:58 +0100, Edmon Chung <[email-masked]>  
wrote:

> However, my point is if "ATOMIC" as a parameter is never transported,  
> then there is no need for a standardized way to create an ACE  
> representation of a UT8 address.  Because each MUA can use any method of  
> transformation to an alt-address they want and the protocol would still  
> function without any problem.  Because in no place would the  
> infrastructure attempt to up/down-convert the address.

Hold on a minute! Suppose, as you say, no atomic parameter is transported.  
If you make such a rule, then you also have to rule whether the behaviour  
then becomes what used to be "atomic=y" or whether it becomes what used to  
be "atomic=n".

I think those of us who are suggesting that the atomic parameter is not  
needed are assuming that behaviour would then become what you get now with  
"atomic=y". In that case, you very much do require a standardised  
downgrade method, and it is essential that it be a reversible method.

OTOH, you seem to be assuming the exact opposite - that the new behaviour  
would correspond to the old "atomic=n", and that leads to all sorts of  
unacceptable consequences, as you have demonstrated.

---

From: Yangwoo Ko <[email-masked]>
Date: Sun, 16 Jul 2006 12:04:07 +0900

Dear Edmon,

I agree with you that having ATOMIC as a flag is not
equivalent to transporting algorithmically encoded
all-ASCII address in an ALT-ADDR slot if;

* we fail to find almost-perfect "in-band" signaling
for EAI suchas "xn--" in IDNA

* and hence it is not safe-to-upgrade

Is this your point?

Regards

---

From: Edmon Chung <[email-masked]>
Date: Sat, 15 Jul 2006 23:26:18 -0400

My main points are I think:

1. if we want to have a standard ACE transformation for local part, then we should keep the ATOMIC parameter (or some whay to convey that message) so that we can actually make use of the standard algorithm to assist in transport to ASCII only users.

2. if we decide not to transport the ATOMIC parameter some way (including specifying it as default), then there is no point having a standard ACE transformation, because everyone can have their own mechanism and it would work the same.

3. I believe there may be some benefit to have a standard ACE transformation, but am not sure.  I think we need further exploration, a couple of people have already suggested that there may be possibility to design a standard two-way convertible ACE for local part... so perhaps if we hear more there...

4. Therefore I think it may be premature to write off the atomic parameter.

Edmon

---

From: Bill McQuillan <[email-masked]>
Date: Sat, 15 Jul 2006 22:22:11 -0700

> 4. Therefore I think it may be premature to write off the atomic parameter.

Let me say it one more time!

LET'S GET RID OF BOTH ALT-ADDR AND ATOMIC.

If there is a guaranteed reversable Local Part ACE which can be recognized
by some sort of flag (like xn-- in IDN) then why is there a need for
anything else?

It has been claimed that there are local parts that an ACE would corrupt,
but I currently do NOT believe that there is enough evidence for this.
Especially if the Local Part ACE only affects portions of a Local Part that
actually are UTF-8 characters. Thus no existing 2821/2 compliant Local
Parts would be changed and we could make recommendations for the creation
of Local Parts in the future that would avoid such corruption.

The problem that I see with Atomic and Alt-Addr is that the data structure
of an email address would be changed to either have a new flag (in the case
of Atomic) or another entire text string (in the case of Alt Addr). Either
of these new data types would need to be handled by everything on the
internet that knows about email addresses. Places such as: 2821bis and
2822bis address fields (obviously), Address books (storage and user
presentation), email addresses used as login id's on many web pages.

Many of these places would need to be enhanced to use this new i18n email
address data type before anyone could be confident in changing to such an
address. On the other hand, if all email addresses can be stored in a form
that is technically allowed by the existing standards merely by prefixing
the "downgraded" version of the address with a flag string, a new i18n
email address can be used as soon as the user's MDA and MUA are enhanced.
The ACEd version may need to be entered manually by some correspondents,
but the final delivery agent is the only one that needs to reverse and
understand the "downgrading".

Now, I know that there is a "rule" that the Local Part is entirely under
the control of the addressee, but it seems to me that reserving a very
unlikely prefix string is a small price to pay for the simplification that
it would bring.

---

From: "Charles Lindsey" <[email-masked]>
Date: Sun, 16 Jul 2006 15:04:55 +0100

> by some sort of flag (like xn-- in IDN) then why is there a need for

Because, for the man without UTF8smtp who wants to send mail to you, your  
ACEd address is bloody incomprehensible. A random collection of characters  
that you will have trouble even copying down onto a piece of paper without  
error, and which you will certainly never carry in your head.

The whole point of current email addresses is that, to a speaker of the  
relevant language, they appear to be meaningful and memorable. The whole  
purpose of this endeavour is to extend that happy state to those whose  
language cannot be expressed decently in ASCII. Now you are asking us to  
accept a system which puts the ASCII speaker in a worse state than the  
non-ASCII people used to be in. And you call that progress?

---

From: Tony Finch <[email-masked]>
Date: Mon, 17 Jul 2006 01:20:41 +0100

> infrastructure attempt to up/down-convert the address.

That isn't true: the down-conversion happens at the MUA when it creates
the alt-address and the up-conversion happens in the MDA when it delivers
the message. These two have to agree on the algorithm, so it must be
standardized.

> When EAIuser1 sends to EAIuser2, even if it provided the ALT-ADDRESS (no
> matter whether it was converted using an algorithm, because without the
> ATOMIC information we cannot assume it is)
>
> WHEN EAIuser after a few weeks tries to initiate an email to EAIuser1
> and ASCIIuserA together, it will not be able to successfully send to
> ASCIIuserA because the header downgrade could not happen for the field
> containing EAIuser1's address.

Why not? EAIuser2 has EAIuser1's alt-address.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
DOVER WIGHT PORTLAND PLYMOUTH BISCAY: NORTHEASTERLY 4 OR 5, BECOMING VARIABLE
3 AT TIMES. FAIR. MODERATE OR GOOD.

---

From: Tony Finch <[email-masked]>
Date: Mon, 17 Jul 2006 01:38:44 +0100

On Sun, 16 Jul 2006, Charles Lindsey wrote:
> On Sun, 16 Jul 2006 06:22:11 +0100, Bill McQuillan <[email-masked]> wrote:
> >

I mostly agree. I particularly agree about the bad impact on non-email
protocols.

> > If there is a guaranteed reversable Local Part ACE which can be
> > recognized by some sort of flag (like xn-- in IDN) then why is there a
> > need for anything else?
>
> Because, for the man without UTF8smtp who wants to send mail to you,
> your ACEd address is bloody incomprehensible.

People already have informal ways of using multiple email addresses, and
these will continue to be sufficient for an ascii user sending to a utf8
user's alt-address, without protocol extensions.

Elegant downgrading of email from a utf8 user to an ascii user is more
interesting, but downgrading to an elegant alt-address can be entirely
handled at the 822 level. The 821 downgrade can be purely algorithmic.
(This would also solve the problem of representing the alt-addresses and
downgradability flags of other email addresses in the envelope, such as
ORCPT parameters.)

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOLE: NORTHEASTERLY 3 OR 4. FAIR. MODERATE OR GOOD, WITH FOG PATCHES AT FIRST.

---

From: John C Klensin <[email-masked]>
Date: Mon, 17 Jul 2006 16:49:37 -0400

Hi.

My apologies -- I was busy enough with other things during the
balance of the IETF meeting that I could sometimes follow this
thread, but not step in.    It seems to me, after reading the
thread,   that two topics are being discussed:

(1) Whether there is an algorithmic mechanism by which any
local-part can be encoded.  If there is -- and some postings in
this thread assume it is -- then any address can be downgraded
using that algorithm and indicating parameters are not needed.
That issue has been debated at length.  It is discussed in
draft-klensin-ima-constraints-00.txt, which is deliberately not
a WG document (but does need upgrading), was discussed when the
WG charter was reviewed, and was discussed extensively before
that.   The summary of the answer is that no such encoding is
possible without violating the "no one interprets addresses
before the delivery MTA" rule and, in some cases, without loss
of information.   There is a bit more flexibility in that
direction when the encoding and decoding are done my MTAs than
in a model that, for many people, seems to follow logically,
which is to do everything in the MUAs, abandoning the notion of
non-ASCII addresses on the wire entirely.  

The reasons for avoiding those approaches from a human interface
and readability standpoint have been discussed by others.  The
technical one is that it cannot be done without loss of
functionality and, for some MTA designs, making this model much
more difficult to implement than it needs to be.   In addition,
it is out of scope: the WG was carefully chartered to follow a
particular approach through to see if it was feasible, including
targeting Experimental documents and implementations, than to
explore the entire possible space of i18n solutions.

Those who are interested in one of the other approaches in that
space are welcome to get organized and propose it.   If it is
simpler and works for as broad a range of cases without
unacceptable constraints, unacceptable assumptions, or
unacceptably bad behavior when older systems (MUAs or MTAs) are
encountered, I assume it will prevail.   But it is not in scope
to try to transform this work into one of those approaches.

(2) View ATOMIC simply as a matter of specifying, presumably
between the originating MUA and submission server, a particular
transformation that would yield only an ALT-ADDRESS in SMTP
transport.   In that model --which is what I heard proposed at
the meeting-- there would be only two cases on the wire:

	(i) ALT-ADDRESS would be specified so that, if a
	downgrade-requiring situation were encountered, the
	ALT-ADDRESS would be substituted as usual.
	
	(ii) ALT-ADDRESS would not be specified.  If a
	downgrade-requiring situation is encountered, the
	message bounces.

In that model, "ATOMIC" (which is probably mis-named for this
case) would be an assertion to the submission MTA that it knew
how to make up an ALT-ADDRESS and place it into the envelope and
should do so.  That strikes me as potentially very useful for
the reverse path address(es), i.e., on the MAIL command and
possibly less so for the forward-pointing addresses.  In the
latter case, the submission MTA probably really doesn't know
what transformation to apply, but that is the case for ATOMIC as
defined in the current documents as well.  And, of course, if
the submission MTA does not take on that role, then there is no
way to specify an ASCII address  in the originating MUA other
than in something that maps directly to ALT-ADDRESS.

There seemed to be considerable support for the second view in
last week's discussion although I'm not sure I consider it
strong enough to be definitive (independent of the need for
ratification on the mailing list).   But I hope we can focus our
discussion on it and not get distracted by various "just convert
local-parts to some agreed-upon ACE always or whenever they
might be needed" approaches.

      john

---

From: Tony Finch <[email-masked]>
Date: Tue, 18 Jul 2006 18:59:34 +0100

On Mon, 17 Jul 2006, John C Klensin wrote:
>
> (1) Whether there is an algorithmic mechanism by which any local-part
> can be encoded. [...] The summary of the answer is that no such encoding
> is possible without violating the "no one interprets addresses before
> the delivery MTA" rule and, in some cases, without loss of information.

It's wrong to say that making all UTF8 addresses algorithmically
downgradable breaks the opacity rule to the same extent as MUA-only IMA.

We can add only one global requirement for local parts which applies only
to UTF8 local parts, which is that they may be downgraded to ASCII
algorithmically. ASCII local parts (downgraded or not) will remain opaque
to everything other than their domain's email agents. In particular,
only the message delivery infrastructure of the domain will ever do the
reverse mapping (not even the users's MUA). The new requirement only
applies to the new kind of local part that the new specifications are
introducing. Existing stuff is not affected.

It isn't clear to me whether the WG is specifying a system that allows
reversible downgrading, such that a UTF8 user can email another UTF8 user
through non-EAI infrastructure. The charter does not seem to include
upgrading but some of the documents discuss it: e.g. -downgrade says an
MUA MUST upgrade downgraded messages. Is this upgrading limited to
replacing MIME encoding with UTF8, or does it include recovery of the
downgraded UTF8 addresses? If the latter, the WG is effectively on a path
towards an MUA-only IMA solution.

With the former (limited) upgrading, it is only capable of matching
impedance between an ASCII sender and a UTF8 recipient, since
upgraded-downgraded UTF8-to-UTF8 email loses information. It's OK to lose
information when downgrading a message from a UTF8 user to an ASCII user
because the lost information (the UTF8 email address) is of no use to the
ASCII user. I see no problem with this.

> transport.

The motivation for getting rid of the ATOMIC option is to simplify the
SMTP extension. If we get rid of it entirely, the ACE-downgradable flag
would be only for the MUA's information, so that it can derive an
alt-address for use when submitting the message over SMTP.

I see no point in keeping the atomic flag for submission only, because
that makes things more complicated, not less!

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HUMBER THAMES: EAST OR NORTHEAST 4 OR 5, OCCASIONALLY 6 IN THAMES. FAIR.
MODERATE OR GOOD.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 19 Jul 2006 10:56:32 +0100

On Tue, 18 Jul 2006 18:59:34 +0100, Tony Finch <[email-masked]> wrote:

> through non-EAI infrastructure.

I think that was one of the scenarios that was regarded as less important  
than the others, though nobody will object if people implement things so  
that it works.

In that case, the final delivery agent may have to upgrade an ACEd address  
(supposing the ACEd form makes no sense to it). If it makes sense after  
upgrading, then it can go ahead with final delivery. If not, or if it  
cannot be bothered to try the upgrade, then it will bounce. But all that  
was already true in the case where an Atomic parameter had been provided,  
so it is hardly new. But I think all other upgrading should be left to the  
MUA, or maybe to the POP3 or IMAP agent.

> Is this upgrading limited to

Not quite, because you still need a UTF8smtp-capable MTA. Yes, there may  
be a way of using our system for MUA-only IMA in particular situations,  
but the whole idea was to make life much simpler for EAI to EAI  
communications (which will be the bulk of the traffic and will just use  
UTF-8 in headers without further thought). But of course most of out  
discussions have been about ameliorating the 5% of traffic where non-EAI  
agents (including senders and recipients) are involved. But don't let that  
tail wag the whole dog.

> alt-address for use when submitting the message over SMTP.

Well I hope I have now shown that a fully reversible ACE using Punycode is  
feasible, provided some care is taken. So that reduces the need for an  
explicit ATOMIC option.
> that makes things more complicated, not less!

Please (I have asked this before) can someone provide a realistic example  
where it would be necessary to use "Atomic=no"? Especially if the ACE was  
fully rebersible.

---

From: Yangwoo Ko <[email-masked]>
Date: Wed, 19 Jul 2006 19:42:22 +0900

Charles Lindsey wrote:
> (snip)
> Well I hope I have now shown that a fully reversible ACE using Punycode 
> is feasible, provided some care is taken. So that reduces the need for 
> an explicit ATOMIC option.

People seem to feel the size of needed care quite differently. If that
"some" is not that big and the implementation of the care in code and
in practice is technically feasible even with mixture of EAI-unaware
components, we may end up with an MUA-only solution discussed at IMAA
mailing list some time ago. Even though this is the case, the pursue
of that solution is clearly out of EAI WG's scope.

---

From: John C Klensin <[email-masked]>
Date: Wed, 19 Jul 2006 07:36:06 -0400

--On Wednesday, 19 July, 2006 10:56 +0100 Charles Lindsey
<[email-masked]> wrote:

>> It isn't clear to me whether the WG is specifying a system
>> that allows reversible downgrading, such that a UTF8 user can
>> email another UTF8 user through non-EAI infrastructure.
> 
> I think that was one of the scenarios that was regarded as
> less important  than the others, though nobody will object if
> people implement things so  that it works.

It was not viewed as "less important".  It was viewed as
unfeasible unless...

	(1) We restrict the range of address formats and uses of
	addresses that were possible in non-ASCII local parts to
	be considerably narrower than those in ASCII local
	parts.  Proposals for such reductions include (i) no
	embedded information at all, (ii) embedded information
	that is separated by standardized delimiters that are
	enumerated by the standard and don't change (probably
	ASCII-only) ones, (iii) embedded information that either
	depends on a list of ASCII delimiters (as in (ii)) or
	that uses a lexical method to identify the delimiters.
	Information embedded using non-delimiter characters or
	subfield lengths known to the receiving system would be
	prohibited by all methods that have been identified.
	
	(2) We impose a de facto restriction on receiving MTA
	implementations that required that they decode the
	transfer encoding of addresses before taking any other
	actions to analyse or process addresses.  Doing things
	this way would be easy and natural for some now-valid
	MTA implementations; it would require completely
	redesigning and reimplementing others in order for them
	to conform.
	
	(3) We require that all implementations support
	upgrading (to UTF-8 addresses) as well as downgrading
	(from UTF-8 addresses to the transport, all-ASCII,
	form), rather than UTF-8 passthrough on a model similar
	to the 8BITMIME model.
	
	(4) We accept the possibility (actually, the certainty),
	that the transport-encoded addresses will leak into
	user-visible contexts.

	(5) We ignore the possibility, nay likelihood, that any
	lexical, in-band, mechanism we devise for encoding an
	address local part will be treated as a valid address in
	some systems.  If a user sends out a UTF-8 address that
	is known to work, and gets back a 550 "no such mailbox"
	code because the address was encoded and no appropriate
	decoder (or alias) existed, he or she is likely to be
	very unhappy... with us.

Now, if our goal was to do something quickly in order to claim
that we had done _something_, those restrictions and
requirements might be plausible.  I suggest that cannot be our
goal: it almost certainly would lead to our having to do a
second design and implementation (or to having someone else do
it) to gain user acceptance.   In particular, each of the
restrictions above should be considered a showstopper:

(1) Effectively makes email with non-ACII local parts a
second-class facility, without the range of capabilities
available in mail with ASCII-only addresses.  I don't consider
that acceptable from a technical standpoint and it would
certainly not be acceptable, long-term, from a cultural or
political one.   Note that since many systems route mail (or
determine how and where they are placed in message stores) based
on information encoded in addresses, getting the MTA out of the
act entirely (as we did with MIME if 7bit transport was used) is
not feasible without even more serious restrictions.

(2) and (3) make the functionality significantly harder to
implement and deploy in practice.  If our goal is rapid
implementation and deployment of conforming and interoperable
implementations, then it is in our interest to keep the
practical implementation costs as low as possible.   We know
that, for a number of significant MTAs that are 8bit clean,
implementation of the basic SMTP extension mechanism with UTF-8
addresses consists largely of disabling existing checks and
restrictions when the option is present.  Turning the
implementation requirement into "complete restructuring" for
even a few implementations is not in our interest unless it is
absolutely necessary, nor is turning downgrade models that
inevitably will require dealing with edge cases in sensitive
code from an option into a requirement.

And (4) and (5) will not be user-acceptable and will just lead
to demands for another approach.  We have already seen problems
with IDNs in which users, expecting to see their own scripts,
have seen punycode instead and objected.  The problem in terms
of user perceptions with email addresses is _much_ worse and,
because of the differences between email model (in which sender
and receiver are basically peers that may not be connected to
each other and that may have different naming conventions) and
the DNS model (in which the registry operator deposits a string
in a database that is queried by the resolver) make it much more
difficult to intelligently report the causes of problems to
users.

>...
> Not quite, because you still need a UTF8smtp-capable MTA. Yes,
> there may  be a way of using our system for MUA-only IMA in
> particular situations,  but the whole idea was to make life
> much simpler for EAI to EAI  communications (which will be the
> bulk of the traffic and will just use  UTF-8 in headers
> without further thought). But of course most of out
> discussions have been about ameliorating the 5% of traffic
> where non-EAI  agents (including senders and recipients) are
> involved. But don't let that  tail wag the whole dog.

I think this is key.  It is also important to remember that the
percentage of cases in which EAI messages will encounter non-EAI
agents will drop over time due to both increasing support for
EAI and entirely predictable user behavior in the presence of
downgrading or bouncing.  Building and carrying around a complex
and expensive arrangement in order to optimize for those cases,
an arrangement that we will need to keep in place even after it
has become unnecessary, is just bad systems engineering.

> Well I hope I have now shown that a fully reversible ACE using
> Punycode is  feasible, provided some care is taken. So that
> reduces the need for an  explicit ATOMIC option.

You have demonstrated no such thing, at least in the absence of
the above restrictions.

regards,
    john

---

From: Tony Finch <[email-masked]>
Date: Wed, 19 Jul 2006 19:08:45 +0100

On Wed, 19 Jul 2006, John C Klensin wrote:
>On Wed, 19 Jul 2006, Charles Lindsey wrote:
>>On Tue, 18 Jul 2006, Tony Finch wrote:
>>>

This will happen if any algorithmic downgrading is part of the spec.

> 	decoder (or alias) existed, he or she is likely to be

This will happen if downgradability is optional.

> In particular, each of the restrictions above should be considered a
> showstopper [...]

Though 4 and 5 are consequences of the current plans.

Your "constraints" document talked a lot about punycoded domains but it
mostly seemed to be arguing that they provided a useful work-around for
untypable or unreadable IDNs, for people who can cope with latin letters
as a lowest common denominator. So I'm not sure it's a total disaster. In
any case, if encoded addresses only appear when exchanging email with
ascii users, and if users can specify a readable alt-address to be used in
preference, then this problem is much smaller.

I agree with your points 1 and 3. I would like some concrete examples of 2:
MTAs that have an architecture which makes decoding ACE local parts to
UTF8 impossible without huge amounts of work.

> It is also important to remember that the percentage of cases in which
> EAI messages will encounter non-EAI agents will drop over time due to
> both increasing support for EAI and entirely predictable user behavior
> in the presence of downgrading or bouncing.  Building and carrying
> around a complex and expensive arrangement in order to optimize for
> those cases, an arrangement that we will need to keep in place even
> after it has become unnecessary, is just bad systems engineering.

You are never going to be able to dispense with downgrade support, if
ESMTP deployment is any indicator. So I think the backwards compatibility
parts of the spec should be as simple as possible. They also need to be
simple enough that your average user can understand them, and if the new
feature is going to be deployable it must not screw up everyone's address
book, directory, browser mailto: handling, etc. etc.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
RATTRAY HEAD TO BERWICK ON TWEED: SOUTHEAST 3 OR 4, OCCASIONALLY 5 AT FIRST.
THUNDERY RAIN OR SHOWERS LATER VISIBILITY: MODERATE OR POOR WITH FOG PATCHES
SEA STATE: SLIGHT

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 19 Jul 2006 19:54:41 +0100

> 	prohibited by all methods that have been identified.

I totally fail to see the need for any such restrictions as you have  
suggested. Please give examples to illustrate the preceived problems.
> 	redesigning and reimplementing others in order for them

They have, as always, the choice of decoding or of bouncing. Bearing in  
mind that it is an EAI-compliant final delivery agent that we are talking  
about, so presumably those are going to accept UTF8-local-parts. No such  
agents exist at the present time, so anyone contemplating providing one  
would do well to start from an existing system into which the decoding  
could easily be incorporated. Clearly, such agents must arrange that any  
local-part starting with "yn--" can never be a valid ASCII local-part in  
their system, but that does not seem an onerous burden.

But if they choose not to implement the upgrade for whatever reason, then  
they must instruct their users to publish Alt-Addresses. Or hey! Maybe  
they instruct their users to include "Atomic=n" (in whatever notation we  
provide), in which case you have possibly found at last a valid need for  
the "Atomic" parameter (though I don't find it a very convincing one).
> 	form), rather than UTF-8 passthrough on a model similar

We require no such thing (though we might strongly encourage it).
> 	that the transport-encoded addresses will leak into

Yes, that is inevitable. But not a show stopper.
> 	decoder (or alias) existed, he or she is likely to be

No, he should be unhappy with his addressee for not giving him an  
Alt-Address, or for subscribing to a service with a less-than-optimal  
implementation. Or he should be unhappy with his own provider for routing  
the message through non-complient systems. Indeed, it is only likely to  
happen where one or other of the endpoints is unable to choose the  
intermediate agents (because one of them is inside some unforgiving  
firewall). But the solution is known, and market forces will soon ensure  
that it is applied where it is needed - no further need for action on our  
part.

> practical implementation costs as low as possible.

The main goal is to provide communication in a natural manner between  
speakers of the same language. Maybe 5% of all communications will have  
one end-point that is non-compliant. I would suppose that less than 0.5%  
would involve compliant end-points with non-compliant agents between. That  
could be a good argument for giving low priority to fixing that scenario,  
but it is not an argument for refusing to provide any mechanism at all.

> downgrading or bouncing.

Exactly. Once users begin to see it working, Market Forces will do the  
rest.

> You have demonstrated no such thing, at least in the absence of

I have demonstrated that Punycode, as defined, will take care of the  
encoding and decoding of any UTF-8 text (<3600 characters) in a reversible  
manner. Those using the model implementation (which goes beyind the strict  
definition of the algorithm) need to be warned of certain pitfalls. There  
is a need to specify some preprocessing (SASL-prep has been suggested, and  
is fine by me). So what have I not demonstrated?