Subject: [EAI] A simple question

From: "Ronald F. Guilmette" <[email-masked]>
Date: Sun, 05 Oct 2014 21:25:03 -0700

Having just today skimmed... for the first time... RFC 6531, I have
a rather basic question.

It appears from my skimming that the first opportunity that either
an SMTP client or an SMTP server gets, under these protocol revisions,
to tell the other that it can support the new internationalized domain
names, is when the server responds to a client-initiated HELO or EHLO
command.  Is that correct?

If so, then I am wondering what sorts of encoding(s), if any, a client
will be allowed to use with respect to the domain name that it may
include within the HELO/EHLO command that it issues, i.e. before
either side of the conversation has said anything definitive about
what it can or can't support.  Must such domain names be represented
as Punycode?

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 6 Oct 2014 05:56:12 +0000

Yes, and yes.

Arnt

---

From: John C Klensin <[email-masked]>
Date: Mon, 06 Oct 2014 09:49:17 -0400

--On Sunday, October 05, 2014 21:25 -0700 "Ronald F. Guilmette"
<[email-masked]> wrote:

> Having just today skimmed... for the first time... RFC 6531, I
> have a rather basic question.
> 
> It appears from my skimming that the first opportunity that
> either an SMTP client or an SMTP server gets, under these
> protocol revisions, to tell the other that it can support the
> new internationalized domain names, is when the server
> responds to a client-initiated HELO or EHLO command.  Is that
> correct?

Yes.  This is normal for SMTP extensions.  Indeed, there aren't
a lot of other choices.  See RFC 5321 and its predecessors if
you are interested.

> If so, then I am wondering what sorts of encoding(s), if any,
> a client will be allowed to use with respect to the domain
> name that it may include within the HELO/EHLO command that it
> issues, i.e. before either side of the conversation has said
> anything definitive about what it can or can't support.  Must
> such domain names be represented as Punycode?

Yes. There is no plausible way for an SMTP extension (this one
included) to extend the syntax of the EHLO command.    As to
HELO, if the client sends that, the EAI extensions just aren't
going to happen so it is a non-issue.

    john

---

From: "Ronald F. Guilmette" <[email-masked]>
Date: Mon, 06 Oct 2014 14:40:38 -0700

In message <[email-masked]>, 
John C Klensin <[email-masked]> wrote:

>included) to extend the syntax of the EHLO command.

Thank you for that info.

Unfortunately, I'm still wondering about that last question.

If the client cannot (yet) know whether or not the server supports
internationalized domain names, and if the client wishes to do
the usual and expected thing, i.e. put its domain name into the
EHLO, then how MUST that domain name be encoded, specifically in
the case where the name of the client is an IDN?

Punycode?

---

From: Mark Martinec <[email-masked]>
Date: Tue, 07 Oct 2014 00:28:46 +0200

> the case where the name of the client is an IDN?

Yes, as answered by Arnt.

RFC 6531 is clear on this:

3.7.1.  The Initial SMTP Exchange

    When an SMTP connection is opened, the SMTP server sends a "greeting"
    response consisting of the 220 reply-code and some information.  The
    SMTP client then sends the EHLO command.  Since the SMTP client
    cannot know whether the SMTP server supports SMTPUTF8 until after it
    receives the response to the EHLO, the SMTPUTF8-aware SMTP client
    MUST send only ASCII (LDH label or A-label [RFC5890]) domains in the
    EHLO command.  If the SMTPUTF8-aware SMTP server provides domain
    names in the EHLO response, they MUST be in the form of LDH labels or
    A-labels.

Btw, the proper name for this encoding is ACE, ASCII-compatible 
encoding.
Punycode is just a name of an algorithm (RFC 3490). Before encoding the
Unicode string must be normalized...

   Mark

---

From: John C Klensin <[email-masked]>
Date: Mon, 06 Oct 2014 18:48:15 -0400

--On Tuesday, October 07, 2014 00:28 +0200 Mark Martinec
<[email-masked]> wrote:

>>>> If so, then I am wondering what sorts of encoding(s), if
>>>> any, a client will be allowed to use with respect to the
>>>> domain name that it may include within the HELO/EHLO
>>>> command that it issues, i.e. before either side of the
>>>> conversation has said anything definitive about what it can
>>>> or can't support.  Must such domain names be represented as
>>>> Punycode?
>>> 
>>> Yes. There is no plausible way for an SMTP extension (this
>>> one included) to extend the syntax of the EHLO command.
>> 
>> Thank you for that info.
>> 
>> If the client cannot (yet) know whether or not the server
>> supports internationalized domain names, and if the client
>> wishes to do the usual and expected thing, i.e. put its
>> domain name into the EHLO, then how MUST that domain name be
>> encoded, specifically in the case where the name of the
>> client is an IDN?
>> 
>> Punycode?
> 

I thought I said that too.  Sorry if either Arnt or I were not
clear.

>     names in the EHLO response, they MUST be in the form of

Yep, seemed pretty clear to me too.   If there is any potential
confusion is would involve failure to look up "A-label" in RFC
5890 or elsewhere.  It implies Punycode encoding for non-ASCII
strings and some additional constraints, such as actual validity
of the IDN label.

> Btw, the proper name for this encoding is ACE,
> ASCII-compatible encoding.
> Punycode is just a name of an algorithm (RFC 3490). Before
> encoding the Unicode string must be normalized...

Mark, you are right about "ACE", but "Punycode encoding" is
correct too.  The reason for using the very precise "A label"
(or "A-label") is that it implies all of the other IDNA2008
constraints of which encoding using the Punycode algorithm and
NFC normalization are just two.

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 06 Oct 2014 19:15:48 -0400

--On Monday, October 06, 2014 14:40 -0700 "Ronald F. Guilmette"
<[email-masked]> wrote:

> wishes to do the usual and expected thing, i.e. put its domain
> name into the EHLO, then how MUST that domain name be encoded,
> specifically in the case where the name of the client is an
> IDN?
> 
> Punycode?

I'm still trying to figure out why you are confused or what you
think isn't clear, so let me add something to my prior note.  In
part because we've observed that too many options make
implementation harder, interoperability problems more common,
and often confuses users, the EAI design tries to allow as few
choices as possible.  In particular:

(1) A conforming i19n SMTP ("EAI") implementation MUST be a
conforming IDNA2008 implementation.  Can't have EAI without IDNA
conformance.

(2) IDNA2008 allows exactly two enocdings, called U-labels (a
label must be a Unicode. native character, string that conforms
to all of the IDNA2008 restrictions) and A-labels (all labels
using characters outside the ASCII repertoire must be in
Punycode encoding).  IDNA2008 also requires that a U-label and
the corresponding A-label be duals of each other: one can
convert back and forth without information loss (if that is not
the case, neither the U-label nor the A-label form is valid).
EAI imposes the added requirement that, on the wire, all
U-labels MUST be encoded in UTF-8.

(3) EAI allows mailbox local parts to be only in UTF-8
(including ASCII).  No other Unicode encoding forms, no ASCII
Compatible Encoding, no tricks with encoded words, etc.

(4) One consequence of the above is that, if U-labels cannot be
used and a domain name contains IDN labels, those labels must be
in A-label form.  The EHLO case (before EAI has been negotiated)
is just a special case of that principle.

(5)  There are technical reasons why it cannot be required, but
EAI strongly prefers homogeneity within domain names -- if there
is more than one IDN label in a domain name, then all of them
should be either A-labels or all of them should be U-labels.
The same principle applies to mailbox names (local-part plus
domain).  Applying the same homogeneity principle to mailbox
names (see (3) above) means that both the local part and the
domain part of a mailbox should be in UTF-8 (i.e., the domain
part uses U-labels if any of its label components are IDNs).

Does that help?
best,
   john

---

From: "Ronald F. Guilmette" <[email-masked]>
Date: Mon, 06 Oct 2014 16:57:58 -0700

In message <[email-masked]>, 
John C Klensin <[email-masked]> wrote:

>I'm still trying to figure out why you are confused

I'm not... anymore.  Somebody answered the question already, so
I'm good.  Thanks for your concern.

{... lengthy technical exposition of all the ins-and-outs snipped... }

>Does that help?

Well, um, see above.

I already got my qustion answered, even before reading your latest
message, so it's all good.

Thank you again.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 7 Oct 2014 04:27:32 +0200

All this assumes that the client has a hostname and knows what it is. These 
days that's not a given.

I think that for a client talking to port 587, I'd just send a constant 
string as hostname. Looking up my own typically NATted address and then 
doing complex work with it strikes me as pointless makework.

Arnt