Subject: Re: [IMA] URIs/IRIs

From: "YAO Jiankang" <[email-masked]>
Date: Wed, 7 Dec 2005 10:19:36 +0800

> ----- Original Message ----- 
> From: "Charles Lindsey" <[email-masked]>
> To: <[email-masked]>
> Sent: Tuesday, December 06, 2005 2:01 AM
> Subject: Re: [IMA] URIs/IRIs
> 
> 
> On Mon, 05 Dec 2005 14:10:33 -0000, John C Klensin <[email-masked]> wrote:
> 
> > That was the key point, and really the only point, I was trying
> > to make.
> >
> > I think some of us should just look at the thing, soon, and get
> > comments to you, rather than tying things up in formalities.
> 
> The most important thing to decide, but one not likely to be clear for  
> some time, arises from the proposal to allow an alternatate delivery  
> address in the RCPT TO: command. Nobody has really indicated how these two  
> addresses get generated. 

after negotiation between sender SMTP and receiver SMTP, if sender SMTP finds that receiver SMTP does not support IEE, the sender SMTP may use alt-address.
in these scenarios, 
 if alt-address value is not specified, the SMTP server can bounce the email to email sender;
if alt-address value is specified,  SMTP may use this alt-address.
The alt-address may be generated by one of the methods below:
 1. the alt-address is specified by email sender.
 2. the alt-address is produced using some algorithms if the I18N address is suitable to be transformed.

>But it would seem desirable to be able to  
> indicate in a mailto: URI "this is my address if you are sending mail to  
> my I18N address, but if your system cannot manage that, then please send  
> to this alterative (but not to both)".
> 
> Now there are many notations in which that might be expressed, but we also  
> need to consider what would appear in the To: and Reply-To: headers of a  
> message, because the same problem arises there too.
> 
> For example, if you want a gold-plated solution, you might invent a new  
> format for <addr-spec>, such as
> 
>      <[email-masked][email-masked]>
> 
> (Harald can suggest more plausible Norwegian examples).
> 
> OTOH, that might be considered too big an upgrade of RFC 2822 for us to be  
> making, though it would lead to an obvious corresponding upgrade for the  
> mailto: URI.
> 
> NOTE: "ålice" above is written in ISO-8859-1, because that is what my  
> mailer does, but clearly the corresponding UTF-8 was intended.
> 
> -- 
> Charles H. Lindsey ---------At Home, doing my own thing------------------------
> Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl
> Email: [email-masked] Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
> PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
> 
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ima
> 

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 07 Dec 2005 10:09:58 -0000

> after negotiation between sender SMTP and receiver SMTP, if sender SMTP  
> finds that receiver SMTP does not support IEE, the sender SMTP may use  
> alt-address.
> in these scenarios,

Exactly. It was case #1 I was concerned about. But how does the email  
sender specify the two possible addresses? Currently, in most MUAs, he  
does it by writing what will become the "To:" header, as in

     To: Ålice <[email-masked][email-masked]>

And if Ålice wants to tell Bøb how to send replies to her, she will  
include in her message

     Reply-To: Ålice <[email-masked][email-masked]>

or she will include, on her website,

     mailto: <[email-masked][email-masked]>

All of which shows that it would be very useful to be able to specify the  
alternate address in To: and Reply-To: headers, whether or not we use the  
particular notation in my examples above.

---

From: John C Klensin <[email-masked]>
Date: Wed, 07 Dec 2005 14:14:01 -0500

--On Wednesday, 07 December, 2005 10:09 +0000 Charles Lindsey
<[email-masked]> wrote:

>...
>> if alt-address value is specified,  SMTP may use this
>> alt-address. The alt-address may be generated by one of the
>> methods below: 1. the alt-address is specified by email
>>  sender.
>>  2. the alt-address is produced using some algorithms if the
>>  I18N   address is suitable to be transformed.
> 
> Exactly. It was case #1 I was concerned about. But how does
> the email  sender specify the two possible addresses?
> Currently, in most MUAs, he  does it by writing what will
> become the "To:" header, as in
> 
> And if Ålice wants to tell Bøb how to send replies to her,
> she will  include in her message
> 
> All of which shows that it would be very useful to be able to
> specify the  alternate address in To: and Reply-To: headers,
> whether or not we use the  particular notation in my examples
> above.

Charles,

While an extended mailto syntax might well work (we would need
to review the syntax carefully), I think it would be a terrible
idea to change the syntax of the address fields in the headers.
The difficulty is related to the one that has already been
explained about "MIME-Version" -- one really cannot use one
header, which might appear in arbitrary order-- to determine how
another one is to be parsed, especially if the new parsing
violates the basic syntax rules of the old one.  And, of course,
trying to parse new headers in the absence of such clues would
make the requirement for some sort of header-marker of the new
syntax even more important.  

So, if this stuff needs to be in the headers, I think you need
to invent some new headers and hope that systems that don't
understand them won't try to parse them at all.  That said, a
different way to look at this would be that any such headers
would be strictly a matter between the MUA and the submission
server, with the latter having the ability to parse them and
turn them into something else if needed.  Please also remember,
in that context, that the convention of sending 822 information
only off to the submission server, and doing it via SMTP, have
never been required.  Some MTAs have always provided different
models for the submission MUAs to communicate with them and have
provided, in those models, mechanisms for specifying envelope
information separately from the header information... rather
than expecting the Submission MTA to guess at the relevant
envelope information from the headers.

   john

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 08 Dec 2005 14:56:57 -0000

> another one is to be parsed,....

One of the properties we are trying to achieve is that no IEE message is  
sent to another agent without first establishing (by looking at the  
capabilities advertised) that agent #2 is able to handle it (e.g. that its  
parser will accept the extended syntax without needing to look at the  
'label' header first).

I grant you that if one invented a header
     Use-Header-Parsing Syntax: A (or B)
where syntaxes A and B were mutually incompatible, then you would have  
real problems (for example if, in MIME Version 2.0, the old Content-Type  
syntax is not a subset of the new). But in the case we are considering  
that would not be so - if you know how to parse the extended syntax then  
you can still parse the old, and if you cannot parse the new, then it  
should never have been sent to you in the first place.

There are indeed dangers in inventing new syntax for <addr-spec>,and we  
would have to consider it very carefully, but the particular problem you  
raise need not be one of them
> understand them won't try to parse them at all.

You mean a new set such as:
    Alt-To:
    Alt-Cc:
    Alt-From:
    Alt-Reply-To
    Alt-Sender:
       and probably some more.

>  That said, a
> turn them into something else if needed.

Yes, if it advertised the IEE capability to the MUA, then it would be  
obliged to downgrade any extended headers (or to bounce) if the next  
server in line could not cope.

But I think the point is that we do have a need for some such capability,  
particularly in the From/Reply-To. But there are many ways we might be  
able to provide it.

---

From: John C Klensin <[email-masked]>
Date: Thu, 08 Dec 2005 16:07:54 -0500

--On Thursday, 08 December, 2005 14:56 +0000 Charles Lindsey
<[email-masked]> wrote:

>> to review the syntax carefully), I think it would be a
>> terrible idea to change the syntax of the address fields in
>> the headers. The difficulty is related to the one that has
>> already been explained about "MIME-Version" -- one really
>> cannot use one header, which might appear in arbitrary
>> order-- to determine how another one is to be parsed,....
> 
> One of the properties we are trying to achieve is that no IEE
> message is  sent to another agent without first establishing
> (by looking at the  capabilities advertised) that agent #2 is
> able to handle it (e.g. that its  parser will accept the
> extended syntax without needing to look at the  'label' header
> first).

Yes.  But one of the arguments for an indicating header field is
apparently that one should be able to use it to determine
whether a message is really in the new form, absent envelope
capabilities.  I don't quite believe that (either?), but, if one
changes the syntax of existing fields, it would require a parser
to handle the headers in two passes --one to look for the
indicator header and one to apply the correct parsing rules to
the fields with legacy names.  That strikes me as a
not-so-wonderful idea.
  
> where syntaxes A and B were mutually incompatible, then you
> would have  real problems (for example if, in MIME Version
> 2.0, the old Content-Type  syntax is not a subset of the new).
> But in the case we are considering  that would not be so - if
> you know how to parse the extended syntax then  you can still
> parse the old, and if you cannot parse the new, then it

But the "new" is a serious error under the old, whether one can
parse it or not.

> There are indeed dangers in inventing new syntax for
> <addr-spec>,and we  would have to consider it very carefully,
> but the particular problem you  raise need not be one of them
>> 

That has been the historical suggestion.  I don't like it very
much either.

> Yes, if it advertised the IEE capability to the MUA, then it
> would be  obliged to downgrade any extended headers (or to
> bounce) if the next  server in line could not cope.
> 
> But I think the point is that we do have a need for some such
> capability,  particularly in the From/Reply-To. But there are
> many ways we might be  able to provide it.

Yes
   john

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 09 Dec 2005 10:55:35 -0000

> But the "new" is a serious error under the old, whether one can

Indeed, but it would be a protocol violation to send a "new" header to  
some agent without first checking it had the capability to process it, in  
which case "garbage in/garbage out" would apply. Of course, when designing  
the protocol we need to satisfy ourselves that it is robust enough for  
such accidents to be rare (but for sure there will always be a few  
rfc-ignorant sites out there).

The 'label' header that is proposed is not intended to be used by agents  
in order to determine what parsing syntax to use (if they know how to  
parse "new" headers at all, then that capability should be turned on  
permanently, because "old" is a subset of "new").

But one of the proper uses for the proposed 'label' header *would* be for  
an agent (which could not tell by other knowledge available to it) to  
decide whether it was necessary to check the capabilities of the next  
agent it was about to contact.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 09 Dec 2005 14:17:05 +0900

>capabilities.  I don't quite believe that (either?),

Just a check: What are, in your view, the reasons for an
indicating header field, if any?

 >But the "new" is a serious error under the old, whether one can

Yes, but I think what Charles is trying to say is that the new
never appears under the old. The reason is that only email
addresses with a truely new address need the syntax that includes
the backup address. To create an example, let's assume that somebody
havs two email addresses, the current one, [email-masked],
and a new one, which I'll denote by [email-masked].
What's in upper-case here is 'true' UTF-8 (i.e. more than just
US-ASCII). Given these two addresses, and [just to take one]
the syntax that Charles has proposed, the following addresses
will exist:
1) [email-masked]
2) [email-masked][email-masked]
3) [email-masked]

1) is just a simple new address
2) is a new address that contains the old address
3) is a simple old address

So the special combination syntax only appears when there is
a new address. Writing something like
[email-masked][email-masked] just doesn't make sense.

Now if we use the rule that in 2), the new form always has to
come first, we don't need to rely on another header or anything
like that to distinguish new and old. The only thing that's needed
is to make sure that 2) can be unambiguosly parsed. For this,
in the specific syntax, we would have to prohibit the character
"|" in new addresses. Because we are going to define exactly
what a new address is, it should not be a problem to exclude "|",
or any other character or syntax we choose as a separator.

If an ASCII character such as "|" doesn't look desirable, we
could also just use a character from Unicode. There's quite
a few of them :-). Of course, in that case, it may not be
easy to type.

Regards,    Martin.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 11 Dec 2005 09:53:29 -0800

Charles Lindsey wrote:
> capabilities advertised) that agent #2 is able to handle it (e.g. that 
> its parser will accept the extended syntax without needing to look at 
> the 'label' header first).

This seems a rather onerous prerequisite.  It places an enormous, 
ongoing barrier to use, given that email does not have any large-scale 
history with requiring prior knowledge about the recipient.  Quite the 
opposite.

That said, folks might want to look at RFC3297, "Content Negotiation for 
Messaging Services based on Email".

 > to handle the headers in two passes --one to look for the

Alas, I had missed the 2-pass impact and the fact the related examples 
do not suffer that bit of overhead.

Helpful exchange.  It is prompting me to formulate a proposal, sent 
under separate cover.

d/

-- 

  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 12 Dec 2005 14:47:17 -0000

On Sun, 11 Dec 2005 17:53:29 -0000, Dave Crocker <[email-masked]> wrote:

> Charles Lindsey wrote:
>> One of the properties we are trying to achieve is that no IEE message  
>> is sent to another agent without first establishing (by looking at the  

Nevertheless, it is the prequisite implied by John's original 'Framework'  
draft and by the draft charter of the proposed WG, and it is the reason we  
are to be charged with producing an Experimental ID, rather than a  
Proposed Standard. Moreover, it was the prequisite for the 8BITMIME idea  
many years back.

If you want to send a message to someone in Chinese, then it is a good  
idea to be sure in advance that he understands Chinese (and has software  
that will cope). Likewise, if you want to send an IEE message with headers  
in UTF-8.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 12 Dec 2005 15:17:16 -0000

> indicating header field, if any?

So that any agent which is asked to handle the message (and that includes  
confused human agents), and cannot tell whether it is IEE just from the  
envelope (if any), can take steps to do any special processing that may be  
needed.

Special processing might include

.Not forwarding it to other agents which did not announce the necessary  
capability.
.Downgrading it
.Bouncing it
.Deciding (inside some upgraded IMAP setup) that it could be delivered in  
UTF-8 rather than UTF-7 (though we have a lot of work to do when it comes  
to proposing an IMAP extension for all this).

> ... To create an example, let's assume that somebody

> a few of them :-). Of course, in that case, it may not be

Hmmm! It seems '|' is a member of <atext> and hence can appear in an atom,  
but
<[email-masked][email-masked]> is still not allowed in a present  
<addr-pec> because it has two '@'s in it. Perhaps `\` would be safer, or  
we could enclose the alternate address in [...] as in  
<[email-masked][[email-masked]]>. It is hardly rocket science to  
devise syntax that would work.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 12 Dec 2005 09:47:45 -0800

>> This seems a rather onerous prerequisite.  ...
> 
> Nevertheless, it is the prequisite implied by John's original 
> 'Framework' draft and by the draft charter of the proposed WG, and it is 
> the reason we are to be charged with producing an Experimental ID, 
> rather than a Proposed Standard. Moreover, it was the prequisite for the 
> 8BITMIME idea many years back.

Charles,

Were this a working group with a charter, then excluding something like 
my suggestion, as being out of scope, would make a lot of sense.

But this is a working group in formation, where consideration of the 
pragmatics constraining particular approaches is more than reasonable.

I cited the MIME/ESMTP (8bitmime) example, too.

I did that because it demonstrated an approach that worked quite well, 
and achieved initial utility with the MIME mechanism.  Meaningful 
infrastructure change, to support the 8bitmime ESMTP option took much 
longer.

d/
-- 

  Dave Crocker

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 Dec 2005 18:58:12 -0500

--On Monday, 12 December, 2005 09:47 -0800 Dave Crocker
<[email-masked]> wrote:

> I did that because it demonstrated an approach that worked
> quite well, and achieved initial utility with the MIME
> mechanism.  Meaningful infrastructure change, to support the
> 8bitmime ESMTP option took much longer.

Actually, I don't think it did.  My recollection of what
happened was that the collection of MTAs that were (i) 8-bit
clean and competent and (ii) came from authors who gave a d**n
about standards started advertising the option as soon as they
added any ESMTP code.  The things that took longer were support
for downgrading and support for 8BITMIME in MTAs that had
internal 7-bit data paths.

   john

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Dec 2005 00:03:40 -0500

--On Monday, 12 December, 2005 14:47 +0000 Charles Lindsey
<[email-masked]> wrote:

>> This seems a rather onerous prerequisite.  It places an
>> enormous,   ongoing barrier to use, given that email does not
>> have any large-scale   history with requiring prior knowledge
>> about the recipient.  Quite the   opposite.
> 
> 'Framework'  draft and by the draft charter of the proposed
> WG, and it is the reason we  are to be charged with producing
> an Experimental ID, rather than a  Proposed Standard.
> Moreover, it was the prequisite for the 8BITMIME idea  many
> years back.

More generally, Dave, it is the condition associated with every
single use of the ESMTP mechanism.  If the next-hop server does
not advertise a capability, one can't use it.  In those cases,
it is not knowledge about the recipient that is required, but
knowledge of the capabilities the next-hop server is willing to
assert that it supports.  And we certainly haven't found it an
"enormous, ongoing barrier to use", at least judging from the
number of implementations that support ESMTP and at least some
options.

> If you want to send a message to someone in Chinese, then it
> is a good  idea to be sure in advance that he understands
> Chinese (and has software  that will cope). Likewise, if you
> want to send an IEE message with headers  in UTF-8.

Strictly speaking, the proposals only require that you be sure
in advance that his systems, and the intermediate ones, can
understand and accept Chinese characters (and/or UTF-8 headers)
without trashing the information.   While that might seem like
an onerous and high-risk requirement (at least to Dave in the
context above), the burden and risk is mitigated considerably by
two things:

	* You would be either mildly confused or mildly insane
	to try to send i18n mail to a submission server that was
	incapable of handling it.  Presumably, the first time
	you tried that, you would discover that it could not
	handle it (the ESMTP option would not be offered and
	your client would reject the message with a protest).
	And you would presumably either get over trying to use
	i18n mail (getting an all-ASCII address from your
	correspondent and using it) or would find a more
	competent submission server.
	
	* Your correspondent and her systems would need to be
	mildly confused or mildly insane to give you an i18n
	address as their contact information unless the
	receiving servers and any intermediate servers they
	configured via MX records were i18n-capable.

Now there are cases, most of them involving mailing list
redistributors, in which the situation gets much more
complicated.  How much more complicated is an open question
which the WG will need to address.  But, for simple
interpersonal mail, and unless there are important cases we
haven't been able to find, "onerous" and "enormous, ongoing
barrier" feel much more like hyperbole than like real issues in
this discussion.

    john

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 13 Dec 2005 07:43:45 +0100

>   configured via MX records were i18n-capable.

Ordinary users don't know what an MX might be.  That's not
mildly insane or mildly confused, that's just as it is :-|

There's some nice ABNF in RFC 4282 for your purposes here
if you replace two FF by F4.
                              Bye, Frank

---

From: Martin Duerst <[email-masked]>
Date: Tue, 13 Dec 2005 18:23:33 +0900

>mildly insane or mildly confused, that's just as it is :-|

Yes, of course. But John is speaking to a technical audience
here. Translated for average users, that means the following:

Your correspondent and her systems would need to be
mildly confused or mildly insane to give you an i18n
address as their contact information unless they are
actually reachable at that address/unless that address
actually works.

That should be easy enough even for average people.
It would also be easy (except for possible spam issues)
to set up a Web page where you can test out that your
address actually works (externally, not only internally).

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 13 Dec 2005 18:18:03 +0900

>
 >This seems a rather onerous prerequisite.  It places an enormous, ongoing 
barrier to use, given that email does not have any large-scale history with 
requiring prior knowledge about the recipient.  Quite the opposite.

I'm not sure about Charles, but what I meant is "hop-by-hop", i.e. for
each single SMTP transaction, not end-to-end (i.e. sender to receiver).
This should be covered by ESMTP options, no need for content negotiation.

Looking at it, it may be a way to deal with the case that an intermediate
relay can't forward an IEE message anymore. An machine-readable,
well-defined format would definitely be preferable to the simple
textual "Undelivered Mail Returned to Sender" type of things used
currently. At least that would allow matching of return messages
to sent mail (and elimination of virus- and spam-related such
messages that go to a pretended sender rather than to the real
sender). There may be other benefits, but I haven't thought this
through fully yet.

Regards,    Martin.

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Dec 2005 07:25:39 -0500

> address actually works (externally, not only internally).

Yes, Martin, thanks.  And the web page is a nice idea.  There is
one more twist to this in the receiving side, and it is an odd
advantage of the addresses actually being in UTF-8 rather than
some more or less idiosyncratic ACE: Email addresses just don't
get created without the intervention of system administrators or
templates or tools installed by system administrators.  For a
sysadmin to permit the creation of i18n addresses for which
there is no server support for for which any MXs are not
properly configured would also constitute either mildly
confusion, mild insanity, or considerable incompetence.   

I can see some scenarios on the sending side that might result
in at i18n address being sent to a non-upgraded server.  For
example, one might have an address that works perfectly well in
my home environment but that gets me into trouble when used with
a submission server provided as part of some meeting-related
network.  But I would figure out, quickly, that unless I don't
use such networks and servers for sending mail, I had better
have alternate addresses available.  On the other hand, such
meeting networks with their own SMTP servers provide some
interesting technical or administrative challenges for a number
of authentication techniques as well, so are not exactly an
increasing trend.

     john

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 13 Dec 2005 18:15:08 +0100

> actually reachable at that address/unless that address

My ISP gave me a "canonical address" and a rule how I
can use this as "vanity host" xyzzy.claranet.de for
[email-masked] addresses (and Message-IDs).

For the finer points like length <= 63 they expect that I
find and read RFC 3696 </joke>

> address actually works (externally, not only internally).

Yes, at least one the infamous O* products has an autotest
feature.  With users you never know, they configure a MX
somehow (for their own domain) and expect that this works.

Worst I've seen in spf-help was a user with "v=spf1 mx -all"
and his MX was 127.0.0.1, yeah.  Asked why his mail causes
FAIL results, IIRC.

SPF might be not the only case where certain kinds of local
parts might end up as domain labels.  For a SOA you'd expect
that a hostmaster knows what he's doing, but generally I'm
pessimistic.   
                             Bye, Frank