Subject: [EAI] Re: localprep and local decision

From: Yangwoo Ko <[email-masked]>
Date: Fri, 01 Dec 2006 20:23:55 +0900

Soobok Lee wrote:
> On Fri, Dec 01, 2006 at 10:16:47AM +0900, Yangwoo Ko wrote:
>>> the nameprep with more delimiters(,=/_ etc) would be enough for
>>> most asian and european email users.
>>>
>>> So, I suggest we have a draft for a stringprep profile for localpart 
>>> encoding in this WG.
>> Section 7 of framework document mentions a little bit about
>> local(?)-prep. I am very glad if you come up with a draft for
>> this.
>>
>> However, I am confused because you mentioned some-prep the
>> local-prep "only" in the context of ACE. I think that the
>> usefulness of local-prep is not necessarily coupled with the
>> use of a specific encoding.
> 
> Let's call it "localprep" until we have a good name for it.
> 
> You are right that localprep needs not be coupled with ACE
> encoding. Its need is very clear even in utf8 localpart.
> 
> As every participants already learned well in the past IDNA effort,
> For example, cafe'(e with accent)@free.fr 
> 
> e' has two form, one is combind e' (U+00e9)
>  the other is two char form: e' (U+0065+U+0301)
> 
> nameprep normalizes the latter form into the former one.
> 
> We need some prohibited lists of chars which seems not
> appropirate in local part, for example (@-like special symbols).
> 
> My gut feeling is that  the exact copy of nameprep is nearly enough
> for localprep , only with some additional prohibited sets of chars 
> and additional set of delimiters other than period.
> 
> But, isn't this vilolating so called "local decision" constraints,
> if we set a rule on how to prohibit and normalize the localpart
> over which the local site admin have "sovereignity" ?

I don't think that using localprep violates anything unless
it is used by MTAs except final delivery SMTP server. For
example, it can used by email clients when accepting user's
input. Or, even it can be used by final delivery SMTP server
to find correct mailboxes for received messages.

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 01 Dec 2006 14:45:12 +0100

> and additional set of delimiters other than period.

At the moment local parts are mixed case, but nameprep removes the
differences between upper and lower case.  Maybe you want saslprep (?)

In another article <[email-masked]> you wrote:

| MUA will be upgraded easily since their market is occupied
| by a few big venders (MS,Mozilla,Opera,Eudora etc).

I doubt that, any "upgrade all MUAs" scheme is IMNSHO dead on arrival.

Frank

---

From: Soobok Lee <[email-masked]>
Date: Sat, 2 Dec 2006 00:10:28 +0900

> to find correct mailboxes for received messages.

But, what if the local site admin wants to maintain separate
mailboxes for  e'  and e+' for some reason. Enforcing 
NFKC-normalization-enabled localprep to localparts would 
bad the admin from having separate mailboxes.

I think applying normalization to localpart are actually
defining a kind of SYNTAXes that should be observed strictly.

Soobok

---

From: Soobok Lee <[email-masked]>
Date: Sat, 2 Dec 2006 00:58:58 +0900

> differences between upper and lower case.  Maybe you want saslprep (?)

Yes. Saslprep has no casefolding, so saslprep may be the model we follow.
I will refer saslprep profile  from now on. Thanks.

> I doubt that, any "upgrade all MUAs" scheme is IMNSHO dead on arrival.

I admit IDNA was adopted by IE7/firefox1.x after 4 years elapsed since
IDNA efforts finished in 2002. It surely took some time. 

But within 2~3 years from now on, IE7+Firefox is expected to  occupy 
95% or more of end user base, i believe.

I expect the similar time for utf8 EAI adoption. Of course, that may vary
on how attractively we standardize this EAI protocol suite for industries.

Soobok

---

From: Soobok Lee <[email-masked]>
Date: Sat, 2 Dec 2006 01:29:39 +0900

> defining a kind of SYNTAXes that should be observed strictly.

I will go further:
 In stringprep RFC  section 2, i found some kind of syntax enforcement:

<quote from section 2>
4) Check bidi -- Possibly check for right-to-left characters, and if
      any are found, make sure that the whole string satisfies the
      requirements for bidirectional strings.  If the string does not
      satisfy the requirements for bidirectional strings, return an
      error.  This is described in section 6.
</quote>
  
<quote from section 6>
 6. Bidirectional Characters
 ...
   2) If a string contains any RandALCat character, the string MUST NOT
      contain any LCat character.

   3) If a string contains any RandALCat character, a RandALCat
      character MUST be the first character of the string, and a
      RandALCat character MUST be the last character of the string.

   Note that requirement 3 prohibits strings such as <U+0627><U+0031>
   ("aleph 1") but allows strings such as <U+0627><U+0031><U+0628>
   ("aleph 1 beh").  [UAX9] goes into great detail about the display
   order of strings that contain particular categories of characters in
   particular sequences.
</quote>

If we apply bidi stringprep rules, aren't we defining some syntax
over localparts?

Soobok Lee

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 04 Dec 2006 00:59:57 +0100

FWIW, the stuff about bidi in the stringprep RFC has turned out to be less 
than optimal.

I believe those restrictions would have been more appropriate at the 
nameprep level (IDNA specific) than at the stringprep level, since they are 
not uniformly useful. But that's water under the bridge.

Check draft-alvestrand-idna-bidi for the details on some issues.

              Harald

---

From: Martin Duerst <[email-masked]>
Date: Mon, 04 Dec 2006 17:37:37 +0900

> the other is two char form: e' (U+0065+U+0301)

I'll refer to these forms as e' and e+' below, same
as Soobok did in another mail.

>over which the local site admin have "sovereignity" ?

The situation for local parts is quite different from
the situation for IDNA. In IDNA, we had to be very careful to
avoid some party registering e' and another party registering e+'.
In EAI, it's the same party being responsible for both, and
a mail administrator would really be crazy if it registered
both e' and e+', other than using e+' as an alias for e'.
A mail administrator would also be rather crazy if s/he
registered only e+', because e' is much more prevalent.

So I think that to a large extent, we can leave things as
advice for mail administrators: When you create an address,
choose one that is at least NFC, better NFKC (and maybe
add some additional advice). Similar advice would be in
force for MUAs, where users type in email addresses from
name cards. On top of that, there will be the advice that
if you want to use a SASL-based security mechanism, you
should stay within certain bounds.

But in both cases, there is no absolute requirement. If
somebody wants to set up an email address with an e+',
it's their choice, they may just be way less reachable
than with e'. If somebody sends email to e+', when it
should be e', it's also their choice, the email will
just bounce. In cases where there are regular bounces,
an administrator may eventually get aware of it, and
if necessary can add an alias e+' -> e'. A very
sophisticated mail receiving agent may also do some
conversions automatically, but that's not necessary.

e' vs. e+' is only one example. For other normalization
mappings, the chance that somebody uses the non-normalized
notation can be higher (e.g. for the full-width equivalents
of ASCII) or lower (e.g. for the number 2 with a circle
around it), but essentially, the thinking can be the same
for most if not all cases.

This means that indeed this can be treated to a large
extent as a local/bilateral issue.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Mon, 04 Dec 2006 17:41:03 +0900

>IDNA efforts finished in 2002. It surely took some time. 

IDNA was in Mozilla about three years ago. That it took so long for
IE7 to appear has nothing to do with IDNA. On the other hand, not
every product gets upgraded at the same speed as Web browsers during
the browser wars, so such timelags are certainly realistic in some sense.

Regards,     Martin.

---

From: Soobok Lee <[email-masked]>
Date: Mon, 4 Dec 2006 19:19:57 +0900

> extent as a local/bilateral issue.

If application of localprep is just a local issue,
 sending MUA clients should not normalize recipients' localparts,
 rather should send as it is entered. Of course, the receiving
 smtp/pop3 server admins can decide either to apply localprep 
 or not to apply localprep to localparts according to their 
 site-specific need. That is, "there is localprep RFC, but, applying
 or not applying this is on your choice".

Do you agree with this advice for MUA/MTA implementors?

Regards,

Soobok

---

From: John C Klensin <[email-masked]>
Date: Mon, 04 Dec 2006 10:55:37 -0500

For whatever it is worth, I think Martin's analysis is exactly
right.  We need to remember that we are operating in an email
environment with an extensive deployed base of users, mailboxes,
and practices not (as with IDNs) looking at an essentially open
field with only the (known) constraints implied by the DNS
itself.  There are lots of reasons why the email environment is
the way it is, most of them good, but, even if there were not,
the deployment base is such that we could not realistically
reopen them without turning this discussion into "new mail
system".

In particular, we have told people for years that making
addresses case-insensitive (whether that is done by algorithm or
aliasing) and aliasing confusing addresses is wise if they want
mail to be delivered.  We have advised sites that want to use
very long local parts to support shorter aliases (provided
either by truncation or by out-of-band communication with
potential senders who are length-limited).  Multiple aliases
that reflect alternate variations on the same name have also
been considered wise: the idea now has to be traded off against
"how much extra spam would you like", but, a few years ago, it
would have been reasonable for my "klensin" mailbox to also be
accessible with local parts containing "john.klensin",
"john.c.klensin", etc., jsut to avoid unnecessary bounces and
out-of-band discussions when people decided to guess.  

We've given out all sorts of advice like this; some written in
RFCs and some not.  But the protocols, for good reason (IMO),
continue to reflect a world in which the range of flexibility in
accepting an address belongs strictly to the delivery host and
system.

Even when looking at POP and IMAP, while it is common to have
the "user" name that is associated with a mailbox be the same
as, or closely-related to, the name of the mailbox as seen by
someone sending traffic to it, that has never been a
requirement.  Some systems have taken advantage of the fact that
it is not and their ability to do that is usually considered A
Good Thing.  Conversely, no system I know of permits, e.g., a
POP mailbox to be accessed using any or all of the "user" names
by which traffic can be sent for delivery to that mailbox... and
that is not something I've every heard people complain about.
It wouldn't be hard to do, it wouldn't violate the protocols,
but no one bothers.

When this WG was chartered, it was agreed that a
"recommendations to mail server operators" document would
eventually need to be written.  it was also agreed that it
didn't belong in the initial charter but that is was something
that should be developed as implementations occurred and
deployment experience accumulated.

However, if people really want to discuss this topic --which
isn't about "localprep" but about advice about as to how mailbox
names (and appropriate aliases and synonyms) should be designed
and developed, I suggest that you start on that document.  I
think that an early start could be immensely useful, but please
do the work by trying to write an I-D, not by speculation and
hand waving about some hypothetical "localprep" that is somehow
like stringprep2003 but not quite.   

My own contribution to that effort is that 

(1) Local mailbox names should be NFKC compliant, at least.

(2) They should also not contain spaces (or any Unicode flavor),
quotes (ditto), or invisible characters if that can be avoided.
And, if it cannot, incoming addresses should be accepted as
matching if those characters are stripped.

(3) Comparisons between the name of the local mailbox and an
incoming local part should be case-independent (and liberal),
independent of the next principle.

(4) For a local mailbox named LocalMailbox, accept any incoming
local part s.t.,
    NFKC(incoming-local-part)=NFKC(LocalMailbox)
or
    NFKD(incoming-local-part)=NFKD(LocalMailbox)

Rules like those are less complex than Stringprep (either
nameprep or SASLprep).  And, for an application which should
probably be motivated by "let's see if the incoming-local-part
is a good match for a mailbox that we have here", that
simplicity is appropriate.

I'd be happy to read and comment on drafts of such a document,
but please don't do the work on this mailing list.

     john

--On Monday, 04 December, 2006 17:37 +0900 Martin Duerst
<[email-masked]> wrote:

>> As every participants already learned well in the past IDNA
>> effort, For example, cafe'(e with accent)@free.fr 
>> 
>> But, isn't this vilolating so called "local decision"
>> constraints, if we set a rule on how to prohibit and
>> normalize the localpart over which the local site admin have
>> "sovereignity" ?
> 
> avoid some party registering e' and another party registering
> e+'. In EAI, it's the same party being responsible for both,
> and a mail administrator would really be crazy if it registered
> both e' and e+', other than using e+' as an alias for e'. A
> mail administrator would also be rather crazy if s/he

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 04 Dec 2006 17:00:07 +0100

> Do you agree with this advice for MUA/MTA implementors?

FWIW I agree.  The sender can't know how the admin at the
site of the receiver partitions the available local parts
if there's no strict rule (like saslprep) to follow.

Frank