Subject: Re: [EAI] mailto:

From: Shawn Steele <[email-masked]>
Date: Fri, 7 Aug 2009 05:15:58 +0000

I agree, but could there be a similar approach that wouldn't break existing mailto handling?

Sent from my HTC FUZE™, a Windows Mobile® smartphone from AT&T

-----Original Message-----
From: Martin J. Dürst <[email-masked]>
Sent: Thursday, August 06, 2009 8:27 AM
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked] <[email-masked]>; Shawn Steele <[email-masked]>; [email-masked] <[email-masked]>
Subject: Re: [EAI] mailto:

The problem with this is that it doesn't work for cc and bcc fields.

Regards,   Martin.

On 2009/08/05 18:36, Alexey Melnikov wrote:
> [email-masked] wrote:
>
>>> From: John C Klensin <[email-masked]>
>>> if one of them contains non-ASCII elements, e.g.,
>>> mailto:[email-masked] or
>>> mailto:[email-masked]
>>>
>>> nothing is lost and some small bits are gained.
>>>
>> I think this method is easy and enough.
>>
>> But I have an another idea to add new option for the mailto.
>>
>> <mailto:[email-masked]&ALT-ADDRESS=ASCII@ASCII>
>>
>>
> That would work as well.
>
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
>

--
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Aug 2009 05:53:33 -0400

--On Friday, August 07, 2009 05:15 +0000 Shawn Steele
<[email-masked]> wrote:

>...

>...

> The problem with this is that it doesn't work for cc and bcc

Shawn,

The problem isn't the approach, it is the fact that mail header
fields are not hierarchical and that there is no way to
explicitly link two fields together as related.  Any syntax that
says, essentially, "this is a header field and its value, that
is a header field and its value,..." is going to run into this
problem.  

What one needs is a way to link the two pieces of information
--the EAI and Alternate addresses -- together as address-pairs,
not separate headers (or something that has the syntax of
separate headers).

So, ignoring niceties of syntax, we could figure out a way to
get 
   mailto:<addr1<addr2>>?cc=<addr3<addr4>>
but it would cause serious problems for existing (legacy) mailto
parsers and applications.

Similarly, one might imagine:
   mailto:addr1|addr2?cc=addr3|addr4
except that "|" is valid in local parts, which might imply a
need for very careful parsing, probably also fouling up some
existing implementations, and noting that a lot of current
mailto implementations are very confused about whether 

   mailto:local-mailbox-name+subaddress@domain

is valid without escaping and treat it as an invalid address;
I'd expect the much-less-frequently-seen "|" to get equally bad
treatment.

I think this is at least part of the reason why the WG
tentatively concluded that inclusion of alternate addresses
would require a separate URI type rather than overloading mailto
(I think incorporating a mailbox syntax that merely had UTF-8
characters in it would not be a problem -- it is, again, the
alternate address syntax that is problematic).  

     john

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 07 Aug 2009 15:08:49 +0100

> parsers and applications.

Not necessarily. If some browser fails to parse is as a valid mailto:, it  
will not even highlight it, so you will not be able to click on it, so you  
will be forced to transcribe it to your MUA manually, which is a nuisance,  
but no huge Big Deal. Ditto if your browser fails to parse it when you do  
cliock on it.

Alternatively, if the existing browser implemention is sufficiently crude,  
it will just pass the whole thing off to whatever MUA it has been  
configured to use, in which case go back to Step 1.

So is there any reqlistic scenario in which some legacy implementation  
might actually try to send it to a non-existent address, or otherwise  
mistreat the message (perhaps even silently)? If not, then I think the  
problem/nuisance can be tolerated.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Joseph Yee <[email-masked]>
Date: Fri, 7 Aug 2009 11:42:12 -0400

mailto also used by mailing list, which reaches many desktop based  
MUA, so it's not only browsers.

-joseph

On 7-Aug-09, at 10:08 AM, Charles Lindsey wrote:

> Not necessarily. If some browser fails to parse is as a valid  
> mailto:, it will not even highlight it, so you will not be able to  
> click on it, so you will be forced to transcribe it to your MUA  
> manually, which is a nuisance, but no huge Big Deal. Ditto if your  
> browser fails to parse it when you do cliock on it.
>
> Alternatively, if the existing browser implemention is sufficiently  
> crude, it will just pass the whole thing off to whatever MUA it has  
> been configured to use, in which case go back to Step 1.
>
> So is there any reqlistic scenario in which some legacy  
> implementation might actually try to send it to a non-existent  
> address, or otherwise mistreat the message (perhaps even silently)?  
> If not, then I think the problem/nuisance can be tolerated.
>
> -- 
> Tel: +44 161 436 6131                         Web: http://www.cs.man.ac.uk/~chl

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Aug 2009 12:25:35 -0400

--On Friday, August 07, 2009 11:42 -0400 Joseph Yee
<[email-masked]> wrote:

> mailto also used by mailing list, which reaches many desktop
> based MUA, so it's not only browsers.

Yes, thanks.

In addition, note that RFC 5335 appears to require the syntax
   <utf8-addr-spec <addr-spec>>

I.e., that leading FWS before the second "<" is not optional.

If we transpose the address directly into a "mailto" URL, that
would imply

    mailto:UTF8-mailbox <ASCII-mailbox>
which doesn't parse at all, leading to a requirement for either
    mailto:<UTF8-mailbox <ASCII-mailbox>>
which EAI-unaware applications would presumably treat as at
least one parsing error, maybe two, or
   <mailto:UTF8-mailbox <ASCII-mailbox>>
creating an exception to the rule that URIs don't need to be
surrounded by brackets or quotes.

So I'm not convinced by Charles's analysis even for browsers.

    john

>>> but it would cause serious problems for existing (legacy)
>>> mailto parsers and applications.
>> 
>>  mailto:, it will not even highlight it, so you will not be
>> able to   click on it, so you will be forced to transcribe it
>> to your MUA   manually, which is a nuisance, but no huge Big
>> Deal. Ditto if your   browser fails to parse it when you do
>> cliock on it.
>> 
>> Alternatively, if the existing browser implemention is
>> sufficiently   crude, it will just pass the whole thing off
>> to whatever MUA it has   been configured to use, in which
>> case go back to Step 1.
>> 
>> implementation might actually try to send it to a
>> non-existent   address, or otherwise mistreat the message
>> (perhaps even silently)?   If not, then I think the
>> Tel: +44 161 436 6131                         Web:
>> http://www.cs.man.ac.uk/~chl Email: [email-masked]
>> Snail: 5 Clerewood Ave, CHEADLE,   SK8 3JU, U.K.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 10 Aug 2009 10:40:37 +0100

>> based MUA, so it's not only browsers.

Indeed, but the same conditions apply. Either
1. The mailing list fails to parse the mailto, and refuses to enter the  
address into the mailing list. But the human get to see there is a problem  
and can try something else.
2. The mailing list accepts the address, and naively attempts to forware  
mail to it as-is. The recipient server either understands it and forwards  
it, of doen't and rejects it.
3. As above, but whatever bounce messages or error responses are generated  
(either by the mailing list, or that server) get lost silently. This is  
the case we want to avoid (because the human is never warned there is a  
problem). So how likely is this case?
> I.e., that leading FWS before the second "<" is not optional.

Was there a reason for that, or it is a bug in RFC 5335?
>     mailto:UTF8-mailbox <ASCII-mailbox>

I would expect it to become
       mailto:UTF8-mailbox%20<ASCII-mailbox>

Actually, life would become much simpler if mailto allowed the syntax
       mailto:<ascii@ascii,domain>

I already find it some problem to remember which protocols expect you to  
type in the <...> and which don't (for example, when typing Message-IDs in  
NNTP - <...> is actually requied in that case, but not in the news URL  
-news:message@id is correct, news:<message@id> is not).

If there were a general relaxation to allow <...> in all comparable URI  
situations, life would become simpler, and generalization to more  
complicated forms of <addr-spec> or ot <message-id> would become much  
easier.

> least one parsing error, maybe two, or

Yes, but I have no problem with parsing errors during changeover from  
legacy software, provided the error is immediately apparent to some  
relevant human.

---

From: John C Klensin <[email-masked]>
Date: Mon, 10 Aug 2009 08:02:39 -0400

--On Monday, August 10, 2009 10:40 AM +0100 Charles Lindsey 
<[email-masked]> wrote:

> 1. The mailing list fails to parse the mailto, and refuses to
> enter the address into the mailing list. But the human get to
> see there is a problem and can try something else.
> 2. The mailing list accepts the address, and naively attempts
> to forware mail to it as-is. The recipient server either
> understands it and forwards it, of doen't and rejects it.
> 3. As above, but whatever bounce messages or error responses
> are generated (either by the mailing list, or that server) get
> lost silently. This is the case we want to avoid (because the
> human is never warned there is a problem). So how likely is
> this case?

Charles,

Even in the less complicated times of long ago, the typical 
behavior of much mailing list software upon finding an 
unparseable mailbox string was to write a log entry and proceed, 
not do something overt and loud to warn anyone.  I know of very 
few people who manage large lists who study the logs every day, 
especially for warnings about trash addresses.  The place this 
sort of problem is most likely to be caught and noticed is in 
the UI by which mailbox strings are added to the mailing list. 
Some of those are fairly dumb, some relatively smart.  The 
options they present differ, and some try to figure out what the 
user meant using heuristics that don't know anything about EAI.

For example, one that I worked with for a while believed that 
there were only kinds of addresses:

   PersonalName WSP <mailbox>
and
   mailbox

and would leave it to delivery time to sort out deliverability

because people routinely omit the quotes when required around 
the PersonalName (and supply them when they are not needed),

   <eai-mailbox <ascii-mailbox>>
would be parsed into
   PersonalName = <eai-mailbox -> "<eai-mailbox"
   mailbox= "<ascii-mailbox>>" -> "ascii-mailbox>"

That might produce a message back to the subscribing user, or 
might not.  It might just produce a confirming message that 
would look ok to someone who was used to EAI addresses and not 
paying a lot of attention (remember, we are talking about users 
here).  It might produce a log entry when the software concluded 
that ">" was not a valid element of a xx21 domain, or might hand 
the string off to the mailing list software (typically by 
writing it into a file) which, in turn, might just drop the 
address containing an invalid domain from the list.

So the odds of a competent human seeing and paying attention to 
a notification are actually not that high.

     john

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 10 Aug 2009 16:37:47 +0200

Charles Lindsey wrote:
> 1. The mailing list fails to parse the mailto, and refuses to enter 
> the address into the mailing list. But the human get to see there is a 
> problem and can try something else. 
Joseph was talking about MUAs using mailto: URLs set by mailing lists, 
not mailto: being part of the command language for mailing list servers.
> 2. The mailing list accepts the address, and naively attempts to 
> forware mail to it as-is. The recipient server either understands it 
> and forwards it, of doen't and rejects it.
> 3. As above, but whatever bounce messages or error responses are 
> generated (either by the mailing list, or that server) get lost 
> silently. This is the case we want to avoid (because the human is 
> never warned there is a problem). So how likely is this case?

---

From: Bill McQuillan <[email-masked]>
Date: Mon, 10 Aug 2009 09:53:05 -0700

> Was there a reason for that, or it is a bug in RFC 5335?

Looking back through the archives, I find that I objected to this on 2007
Mar 25:

   http://www.ietf.org/mail-archive/web/ima/current/msg01621.html

but when there was a response about vague concerns for parsing, I
apparently dropped the ball and did not press the point. Mea culpa.

Looking farther back, I can't quite find where this idea came from,
although it might have been inherited from earlier iterations where the
brackets for the <ascii-addr> were "{}" and there was concern that they
were included in <atext>, but this would only have been a concern when they
were outside of the <angle-address>.

Final verdict (IMHO): bug in REFC 5335.

-- 
Bill McQuillan <[email-masked]>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 11 Aug 2009 06:47:09 +0200

> apparently dropped the ball and did not press the point. Mea culpa.
This was (if my faulty memory is reconstructing this correctly) based on 
other experience with "optional" whitespace, such as the whitespace 
after the colon in a header line (NetNews ignored the rule that it can 
be omitted for so long that it had to require it when standardization 
came around) or the allowed whitespace inside an email address (harald  
(comment)  @    alvestrand   .  no is supposed to work); as chair, I did 
not see a compelling argument that not requiring the whitespace resulted 
in any value to the user, so it stayed required.

(The <<>> format seems to have been created around February 2007, so the 
arguments were supposedly fresh in people's minds when your comments 
were discussed on the list - the headers doc went to Last Call soon 
after, and the issue was not re-raised there, so I don't know if we have 
more recorded data on this issue.)
> Final verdict (IMHO): bug in REFC 5335.
I think it's "Working as intended".

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 11 Aug 2009 10:46:53 +0100

> I think it's "Working as intended".

But not if its obvious transcription into a mailto (if we decide that is  
desirable) is going to lead to some non-standard URI syntax). N.B.,  
removing redundant CFWS from an <addr-spec> when constructing a mailto may  
well require attention in the present setup, but implementations should  
not be placed in the position of having to remove _REQUIRED_ stuff from an  
<addr-spec) (unless it was required for some very good reason).

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 11 Aug 2009 11:07:44 +0100

On Mon, 10 Aug 2009 13:02:39 +0100, John C Klensin <[email-masked]> wrote:

> Even in the less complicated times of long ago, the typical behavior of  
> much mailing list software upon finding an unparseable mailbox string  
> was to write a log entry and proceed, not do something overt and loud to  
> warn anyone.

Sure, but much existing software does stupid/unhelpful things all the  
time, and the sky does not fall in.

So some legacy piece of software (whether for mailing lists or anything  
else) which is asked to process an <addr-spec>, for whatever reason, may  
do any of the following

1. Silently accept and then mishandle an ASCII <addr-spec> not conforming  
to RFC 5322 (e.g. a mismatched quoted-string in its local part).

2. Silently accept and then mishandle a "utf-8@utf-8" <addr-spec>.

3. Silently accept and then mishandle a "<utf8@utf8 <ascii*ascii>>".

Case 1 already happens. Case 2 is certainly going to happen, and we will  
have to live with it.

So the _real_ question to ask is whether the consequences of allowing Case  
3 are so horribly worse than the inevitable (and existing) consequences of  
the first 2 that we should seek to forbid Case 3 entirely. That is a case  
that has to be made, and it has not been made yet.

But bear in and that legacy software is going to be around for many years  
to come in places where nobody ever encounters such a thing as Non-Ascii.  
So long after we find we no longer need to worry about all the more  
obscure downgradings, we may still find it essential to have alt-addresses  
available to us (at least in From/Reply-To headers). Or maybe not ...

That is the question. Will we be wors off, or better off, without it  
(given that something is not going to work either way).

---

From: John C Klensin <[email-masked]>
Date: Tue, 11 Aug 2009 08:02:17 -0400

--On Tuesday, August 11, 2009 11:07 +0100 Charles Lindsey
<[email-masked]> wrote:

>...
> So the _real_ question to ask is whether the consequences of
> allowing Case 3 are so horribly worse than the inevitable (and
> existing) consequences of the first 2 that we should seek to
> forbid Case 3 entirely. That is a case that has to be made,
> and it has not been made yet.

Actually, because this is a new feature and new syntax, I
suggest that the case which has to be made is that it is safe
and worth the trouble.  In other words, those who are hesitant
about it should not be obligated to prove that it will not be
"horribly worse".

> But bear in and that legacy software is going to be around for
> many years to come in places where nobody ever encounters such
> a thing as Non-Ascii. So long after we find we no longer need
> to worry about all the more obscure downgradings, we may still
> find it essential to have alt-addresses available to us (at
> least in From/Reply-To headers). Or maybe not ...

Yes, and we don't know how those legacy systems will behave when
confronted with "new" syntax.  We can safely predict that some
will fail in a reasonable fashion and that others will behave
badly.  We can predict that there will be some (perhaps many)
good quality implementations of in-transit downgrading and some
bad quality ones.  We can also predict that there will be some
firewall/gateway issues with new headers and new syntax (because
those systems have a different mission and because,
historically, most of them have been of poor quality and very
slow to follow new SMTP extensions).

> That is the question. Will we be wors off, or better off,
> without it (given that something is not going to work either
> way).

Actually, as some level, no one is disagreeing with you,
depending on what you mean by "it".   One version of "it" is
"change the syntax, supporting downgrading with forward and
backward-pointing addresses".   That is beginning to feel like
"worse off", both because of the cases that won't work and
because it is a significant change with ripple side-effects
(including the complications with "mailto:")..

The other is some version of what Ernie, Yao, and I are
suggesting (I won't know if our suggestions are completely
consistent until someone writes an I-D).  That suggestion is:

(1)  We drop forward-pointing downgrading on the grounds that
(i) virtually all of the cases are configuration errors or
general stupidity, (ii) it eliminates the complex "all of the
addresses have to be downgradeable to preserve the recipient
list" rules, (iii) it eliminates the interdependent header
logic, which we know to be fragile.

(2) We drop the <foo <bar>> syntax as (i) too radical a change
for a transition mechanism, (ii) raising excessive opportunities
for security-related mischief, (iii) complicating MUA-Submission
Server interactions, (iv) complicating mailto, and (v)
representing a higher-than-necessary risk of "interesting"
foul-ups in legacy systems as well as totally confusing lusers
who haven't seen the syntax before.

(3) We preserve the alt-address parameter on the SMTP MAIL
command, while dropping it on RCPT (as the result of (1)).
That implies that we retain a fallback to an ASCII address for
NDNs and MDNs.  Of course, that doesn't help with the blowback
problem, and might make it a tad worse.  But the problem is
really not different from the single-address one -- when
possible, rejections at SMTP time are probably better than
sending NDN messages back and, if one is going to send an NDN,
being reasonably certain that the address to which it is sent is
valid for the sender is a good idea.

(4) We encourage the use of 
   From: addr1, addr2
as a representation of 
   MAIL FROM:<addr1>, ALT-ADDRESS="addr2"
Doing so requires that we accept the fact that the semantics are
a little different and the observation that multiple "From:"
fields are not well supported.   We may have to encourage better
support, think about whether clarifying language in RFC 5322bis
is needed about what replies to such messages means, and so on.
But support for multiple From-address syntax has been specified
and required for many years (so no new syntax).  I note that
there is, properly, no multiple-address syntax for "Sender:",
but think we can live with that.   Using "From:" that way really
is a transition mechanism-- when it ceases to be useful, it
disappears.

Part of my reasoning is based on an assumption that may not be
true but that is based on a certain amount of experience.  That
is that a legacy system that encounters 
   From: EAI-addr, ASCII-addr

is much more likely to behave reasonably --since it knows that
there are two addresses, whether it "likes" them or not -- than
a legacy system that encounters

   From: <EAI-addr <ASCII-addr>>

In the first case, the EAI-addr may be rejected as trash or
turned into a name phrase or comment, but either the second one
will go through or there will be a comprehensible error message.
It is even possible that both addresses will go through because
the MUA and MSA fail to insist on ASCII addresses.   In the
second, there is, IMO, considerable risk of the whole address
being rejected or trashed beyond recognition.   And, once the
patterns of "trash beyond recognition" are understood by the bad
guys, there are possibilities of interesting security exploits
with it.

    john

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 12 Aug 2009 12:53:26 +0100

> about it should not be obligated to prove that it will not be

It should not be "horribly worse" in the case of straightforward attempts  
to deliver mail, since if our protocola are implemented correctly, no  
sample of a header containing <utf8@utf8<ascii@ascii>> should ever reach a  
legacy server that does not advertise UTF8SMTP. The risks all arise from  
the umpteen pieces of ancillary software (mailing list expanders, address  
lists, and the like) which tend to get written with less attention to the  
details of the standards.

But I think this thread has now established that it is really a choice  
between two alternatives, neither of which will be 100% perfect. But your  
message contains a good analysis of the possibilities, so we should work  
on that.

My own opinion is that we should stick with what we have got, and not rush  
into further attempts to get it onto the standards track at this time.

OTOH, we should listen carefully to the Chinese, who are about to  
standardize our experimental protocols for their own use. To what extent  
can the Chinese standards be undone if we ultimately decide to standardize  
something slightly less ambitious? And how bad would it be if we decided  
to exclude that extra header syntax whilst the Chinese standard still  
allowed it?

> suggesting ...
> (1)  We drop forward-pointing downgrading...

Except that I doubt it is the "forward pointing" stuff that is going to  
break.
> who haven't seen the syntax before.

Yes, that is the main change proposed.
> valid for the sender is a good idea.

But I think it would be bizarre to allow an ALT parameter in MAIL, but not  
in RCPT. That is a recipe for confusion.
> fields are not well supported. ...

That is a messy solution, but I agree it might work.