Subject: [EAI] Punycode

From: Shawn Steele <[email-masked]>
Date: Tue, 7 Oct 2014 19:06:39 +0000

> (1) A conforming i19n SMTP ("EAI") implementation MUST be a
> conforming IDNA2008 implementation.  Can't have EAI without IDNA
> conformance.

I think that depends on the implementation.  EAI allows UTF-8 for most everything.  As pointed out it's difficult to complete the SMTP handshake without having the punicode, however there are many other mail related tasks that can be done without needing to know about IDNA?

For example, an application needn't send mail itself, it could gather the information it needed and then use a compliant STMP library to actually send the message.  In that case the application itself may not need to know anything about IDNA itself.

Indeed, I'd prefer that applications use IDN/DNS/Mail APIs that abstract away some of these details.

-Shawn

---

From: John C Klensin <[email-masked]>
Date: Tue, 07 Oct 2014 15:23:40 -0400

--On Tuesday, October 07, 2014 19:06 +0000 Shawn Steele
<[email-masked]> wrote:

>> conforming IDNA2008 implementation.  Can't have EAI without
>> IDNA conformance.
> 
> I think that depends on the implementation.  EAI allows UTF-8
> for most everything.  As pointed out it's difficult to
> complete the SMTP handshake without having the punicode,
> however there are many other mail related tasks that can be
> done without needing to know about IDNA?

Sorry... difference in terminology rather than a disagreement
about substance.  When I said "implementation", I was referring
to the whole package including the actual application code and
any libraries, services, or utilities it called on.  I think you
are talking, about what I would call the direct/ immediate
application code and not the collection.  In your vocabulary, I
would have said "an i18n SMTP implementation, including all of
the relevant libraries, MUST conform to IDAN2008 requirements"
or words to that effect.

In defense of my choice of wording, if you implemented something
that called on, e.g., DNS resolution libraries that did not
conform to IDNA and that got the user into trouble or unexpected
behavior, your responding to the complaint with "no me, I just
used that library over there" might impress your management as
they tried to figure out where to assign responsibility for the
fix but would almost certainly not impress the user.

 best,
   john

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 7 Oct 2014 21:37:55 +0200

On Tuesday, October 7, 2014 9:06:39 PM CEST, Shawn Steele wrote:
> for most everything.  As pointed out it's difficult to complete 
> the SMTP handshake without having the punicode, however there 
> are many other mail related tasks that can be done without 
> needing to know about IDNA?

I conform without IDNA.

What I send as EHLO argument isn't my hostname, which is a tiny little bit 
against a rule from, IIRC, 1123. But who cares about that rule? 34.7% of 
clients that connect to port 587 think their hostname is macbook.local and 
another 27.9% send "ehlo 192.168.42.42", so that rule is pointless anyway. 
You can send "ehlo outlook-3-14.microsoft.com" if your version number 
happens to be 3.14, and that'll be more informative than that "ehlo 
macbook.local".

Arnt

---

From: John C Klensin <[email-masked]>
Date: Tue, 07 Oct 2014 15:42:43 -0400

--On Tuesday, October 07, 2014 21:37 +0200 Arnt Gulbrandsen
<[email-masked]> wrote:

> What I send as EHLO argument isn't my hostname, which is a
> tiny little bit against a rule from, IIRC, 1123. But who cares
> about that rule? 34.7% of clients that connect to port 587
> think their hostname is macbook.local and another 27.9% send
> "ehlo 192.168.42.42", so that rule is pointless anyway. You
> can send "ehlo outlook-3-14.microsoft.com" if your version
> number happens to be 3.14, and that'll be more informative
> than that "ehlo macbook.local".

As long as the "hostname" is all-ASCII --whether an IP address,
something like macbook.local, etc., IDNA is irrelevant (or, if
you prefer, you conform to it because you are not using IDN
labels).   So, unless I misunderstand you, you are conforming
_to_ IDNA (which specifies how labels with non-ASCII characters
in them are to be handled) even if you are not _using_ IDNA.

You may or may not conform to SMTP (or care whether you do or
not), but neither IDNA nor EAI are part of the picture above.

    john

---

From: Shawn Steele <[email-masked]>
Date: Tue, 7 Oct 2014 19:45:46 +0000

Sorry, I was also missing "SMTP" (somehow, I explicitly looked for it :)   And thinking of more general EAI aware apps that may not even deal directly with that.  SMTP specific apps would likely need to know more.

Also, it may be 'good' for some apps to be aware of IDNA even if they don't really need to know about it.  Then if they see Punycode when they expected to see Unicode they may be able to do something to be more robust.

-----Original Message-----
From: John C Klensin [mailto:[email-masked]] 
Sent: Tuesday, October 7, 2014 12:24 PM
To: Shawn Steele; [email-masked]
Subject: Re: [EAI] Punycode

--On Tuesday, October 07, 2014 19:06 +0000 Shawn Steele <[email-masked]> wrote:

> I think that depends on the implementation.  EAI allows UTF-8 for most 
> everything.  As pointed out it's difficult to complete the SMTP 
> handshake without having the punicode, however there are many other 
> mail related tasks that can be done without needing to know about 
> IDNA?

Sorry... difference in terminology rather than a disagreement about substance.  When I said "implementation", I was referring to the whole package including the actual application code and any libraries, services, or utilities it called on.  I think you are talking, about what I would call the direct/ immediate application code and not the collection.  In your vocabulary, I would have said "an i18n SMTP implementation, including all of the relevant libraries, MUST conform to IDAN2008 requirements"
or words to that effect.

In defense of my choice of wording, if you implemented something that called on, e.g., DNS resolution libraries that did not conform to IDNA and that got the user into trouble or unexpected behavior, your responding to the complaint with "no me, I just used that library over there" might impress your management as they tried to figure out where to assign responsibility for the fix but would almost certainly not impress the user.

 best,
   john

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 7 Oct 2014 21:57:23 +0200

> in them are to be handled) even if you are not _using_ IDNA.

I suppose you might say so. Put that way it's wonderful: In one stroke I 
manage to comply with IDNA, X.400 and Fidonet too!

> not), but neither IDNA nor EAI are part of the picture above.

What I tried to say, again: It's perfectly practical for a port-587 client 
to send EAI mail without ever using IDNA. Doing so requires one SMTP rule 
violation, but that SMTP rule is broken so often by so many people that it 
has lost all force.

Arnt

---

From: "Ronald F. Guilmette" <[email-masked]>
Date: Tue, 07 Oct 2014 13:20:25 -0700

In message <[email-masked]>, 
Arnt Gulbrandsen <[email-masked]> wrote:

>against a rule from, IIRC, 1123. But who cares about that rule?

I do.  So do some other people I know.

For some time now I have had turned on the feature(s) in Postfix that
provide checking of the HELO/EHLO string.  I personally check it to
make sure that it is a _not_ in the form [A.B.C.D] (or if it is, then
I reject the command and terminate the SMTP session) and also, using
a built-in feature of Postfix, I allow Postfix to check to see if
the domain name given can be resolved to an IP address.  If not,
then also in this case the SMTP client will receive a 5xx response
and be told to go away.

These two checks, together get rid of a LOT of spam.

>34.7% of 
>clients that connect to port 587 think their hostname is macbook.local

Well, yes, but now you're talking about trusted/validated/local submission 
clients, as opposed to stuff coming in from the Internet at large.  So
that's a whole different case, and one for which rigorous checking of
HELO/EHLO strings is, for the reason you have noted, inappropriate.

---

From: John C Klensin <[email-masked]>
Date: Tue, 07 Oct 2014 16:40:12 -0400

--On Tuesday, October 07, 2014 19:45 +0000 Shawn Steele
<[email-masked]> wrote:

> Sorry, I was also missing "SMTP" (somehow, I explicitly looked
> for it :)   And thinking of more general EAI aware apps that
> may not even deal directly with that.  SMTP specific apps
> would likely need to know more.
> 
> Also, it may be 'good' for some apps to be aware of IDNA even
> if they don't really need to know about it.  Then if they see
> Punycode when they expected to see Unicode they may be able to
> do something to be more robust.

Definitely.  But that is, it seems to me, to be a separate issue.

    john