Subject: [apps-discuss] Last minute concern on nullmx draft

From: "John Levine" <[email-masked]>
Date: 4 Sep 2014 18:04:26 -0000

Wietse Venema and Viktor Dukhovni just noticed the nullmx draft (sigh)
and have concerns about using a 521 return code on RCPT TO.

They didn't explain it very clearly, but now I see that the problem is
that 221, 421, and 521 currently always mean that you should
disconnect now. We're proposing a context where it just means that an
address can't be delivered but the SMTP session is otherwise OK.  They
argue that this makes it harder to write milters and such.  This seems
like a plausible concern.

So I'd suggest changing the 521 return code to 550 (Requested action
not taken: mailbox unavailable) or 553 (Requested action not taken:
mailbox name not allowed) instead.

I realize this is pretty late in the process, but it seems like a
reasonable change to make.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 4 Sep 2014 15:26:54 -0700

> like a plausible concern.

Interesting observation.  Is that by design, or was it a coincidence?

Why does it make milters harder to write?

Suggested resolution seems OK to me.

-MSK

---

From: Nico Williams <[email-masked]>
Date: Thu, 4 Sep 2014 17:43:22 -0500

> Interesting observation.  Is that by design, or was it a coincidence?

The same codes are not assigned for the 2xx/4xx/5xx ranges, but at
least for x21 they are.

> Why does it make milters harder to write?

http://www.ietf.org/mail-archive/web/ietf/current/msg89507.html

The idea is that 5xx can be mapped to 4xx:

| Until now the Postfix community has relied on this property as part
| of a temporary safety net (to avoid missing email, replace 5XY with
| 4XY; after watching the logs for some time, remove the safety net).

Seems more than plausible to me.

Nico
--

---

From: "John R Levine" <[email-masked]>
Date: 4 Sep 2014 20:28:47 -0400

> Why does it make milters harder to write?

Beats me, I still use qmail and milters are this strange exotic thing.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: Steve Atkins <[email-masked]>
Date: Thu, 4 Sep 2014 17:33:55 -0700

>>> like a plausible concern.

553 seems like a reasonable change.

> | 4XY; after watching the logs for some time, remove the safety net).

Is this documented anywhere? (Should it be?)

Cheers,
  Steve

---

From: Ned Freed <[email-masked]>
Date: Thu, 04 Sep 2014 20:01:56 -0700 (PDT)

> Is this documented anywhere? (Should it be?)

As I pointed out previously, since the mapping fails in a large number of
cases, either resulting in undefined codes or a semantically distinct code, I
certainly hope not.

As for what actually is documented, the theory of reply codes in RFC 821 only
states that the third digit is reserved for "the finest gradation of
information".

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 04 Sep 2014 20:13:01 -0700 (PDT)

> Seems more than plausible to me.

The concept of treating a 5YZ as temporary is entirely plausible, the specific
implementation of blindly mapping codes around is not.

And it still doesn't explain why this makes milters hard to write.

				Ned

---

From: Tony Finch <[email-masked]>
Date: Fri, 5 Sep 2014 11:21:36 +0100

John Levine <[email-masked]> wrote:
> address can't be delivered but the SMTP session is otherwise OK.

I think they are right that this is not following the theory of reply
codes properly.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Trafalgar: Cyclonic in northwest, otherwise mainly northerly or northwesterly
5 or 6. Slight or moderate. Showers in northwest. Good.

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Sep 2014 10:54:10 -0400

--On Thursday, September 04, 2014 18:04 +0000 John Levine
<[email-masked]> wrote:

>...
> They didn't explain it very clearly, but now I see that the
> problem is that 221, 421, and 521 currently always mean that
> you should disconnect now. We're proposing a context where it
> just means that an address can't be delivered but the SMTP
> session is otherwise OK.  They argue that this makes it harder
> to write milters and such.  This seems like a plausible
> concern.
> 
> So I'd suggest changing the 521 return code to 550 (Requested
> action not taken: mailbox unavailable) or 553 (Requested
> action not taken: mailbox name not allowed) instead.

Hi.

Replying to this message for convenience, but I have read the
"Last minute concern" thread to date.

John wrote me on Tuesday suggesting that I fix
draft-klensin-smtp-521code to explicitly allow for the RCPT
case.   I had intended to get that done immediately but was then
distracted by another set of issues.  Turns out that was good
luck... and better luck that I decided to read this thread
before posting that revision.

All other things being equal, I prefer not using 521 for a
situation in which an SMTP client concludes from external
information (nullMX included) that a site won't accept mail.  

My interpretation of the SMTP model is that the only way one can
definitively figure out that a server won't accept something is
to ask the relevant server.  This is true whether the thing
being proposed is a domain name/ host name, a mailbox (and hence
a local-part), an extension, or a command (e.g., VRFY or EXPN).
That doesn't argue against nullMX, but, especially in the light
of many years of experience with disconnects between mail/SMTP
administrators and DNS administrators, it strongly suggests that
we should distinguish between direct information and indirect
information.  

Given the theory of second-digit reply codes, I also believe
that 55x is a better choice for the RCPT case than 521.  But I'm
also reluctant to see anything in the 550 - 554 range used,
partially because of the "indirect information" issue outlined
above. 

So, if we have to invent or assign a different code, why not
invent 556 or, in principle, 522?   I/we have the "code 521"
draft open; it would take only a few extra minutes to add a new
code to it with very specific functionality of either "the DNS
told me there was no service for that domain" or "I know from
external sources, such as the DNS, there there is no service".

I will hold draft-klensin-smtp-521code-02 until someone tells me
what to do.  If 521 is not needed for nullMX, we don't need
additional codes, it is probably ready to go to Barry and Pete
as an independent submission.  

     john

p.s. In working on draft-klensin-smtp-521code, I discovered
another glitch in nullMX; watch for a separate note.

p.p.s. "[email-masked]" is not a deliverable address.

---

From: "John R Levine" <[email-masked]>
Date: 5 Sep 2014 10:59:10 -0400

> external sources, such as the DNS, there there is no service".

556 is fine with me if it's fine with everyone else.  I'm assuming that if 
your draft invents it, we can just refer to it in nullmx, right?

Regards,

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Sep 2014 11:32:24 -0400

Hi.

In the process of working on nullMX-related text for
draft-klensin-smtp-521code, I realized that there is a missing
piece that the spec should address.  It is a little more complex
than trying to pick an error code.

Even without extensions, SMTP has three commands whose arguments
are forward-pointing addresses, not just RCPT.  The dummy server
approach of RFC 1846 (etc.) deals with all of them the same way
-- if the host won't accept an SMTP connection, then RCPT, VRFY,
and EXPN are equally irrelevant.  But, if the client is scanning
for commands and doing DNS lookups, things get more complicated.
In the RCTP case, there is a clear specification about doing MX
lookups on the domain in the address -- the facility on which
nullMX depends.  But there is no such specification or
requirement to do MX lookups for VRFY and EXPN.  At least by
implication, the server with which the mail session is open is
expected to figure out how to respond to the command with
information or indicate that it cannot (with 252, 251, or 551 --
See RFC 5321 Section 3.5.3).

The other thing that is notable about this is that 252 is a kind
of "indirect information" response, specifically that the host
contacted will try to deliver a message sent to that address,
but has no information as to whether it is valid.

The nullMX spec is correct as written because it talks about
"mail delivery" and "mail delivery attempts" but phrases like
"without requiring domains to create SMTP listeners dedicated to
preventing delivery attempts" strongly imply that SMTP listeners
are unnecessary to prevent timeout and potential retry behavior.

It seems to me that the nullMX spec is incomplete if it does not
address this issue in some way, even if all it says is that the
DNS entry provides protection against actual mail transactions
but not, e.g., query transactions such as those implied by VRFY
or EXPN.  In particular, a client can conform to the nullMX spec
but still try to open SMTP connections to the relevant target
host.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Sep 2014 12:03:30 -0400

--On Friday, September 05, 2014 10:59 -0400 John R Levine
<[email-masked]> wrote:

>> draft open; it would take only a few extra minutes to add a
>> new code to it with very specific functionality of either
>> "the DNS told me there was no service for that domain" or "I
>> know from external sources, such as the DNS, there there is
>> no service".
> 
> 556 is fine with me if it's fine with everyone else.  I'm
> assuming that if your draft invents it, we can just refer to
> it in nullmx, right?

Absolutely -- no more problem referring to it than there was
with 221 (or a bit less given our exchange at the beginning of
the week).  If others like that, I'll get a new draft with it
specified out RSN.

The ADs and this WG should figure out, ideally very soon,
whether it wants to adopt/review that draft or whether I should
specify discussion on ietf-smtp and handle it as an individual
submission.
 
    john

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 5 Sep 2014 10:12:02 -0700

> Seems more than plausible to me.

But this seems to me like it's building something based on a coincidence,
which is never a safe thing to do.

I suppose since there's widely deployed code that does it, then that's
something worth considering.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 5 Sep 2014 17:24:04 -0000

> In particular, a client can conform to the nullMX spec

I don't see how this can happen.  If a domain publishes nullmx, there
is no target host.

Perhaps I'm misreading 5321, but it's always been my understanding
that VRFY and EXPN only apply to domains for which the current host is
a delivery host.  If I am foo.com, not an MX of bar.com, and you say
"VRFY [email-masked]" or "EXPN [email-masked]", I'd return 550 or maybe a
passive aggressive 252.

How would nullmx change this?

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Fri, 05 Sep 2014 10:38:44 -0700 (PDT)

> nullMX depends.

There are actually two cases here - doing the lookup at RCPT TO
time and doing it later. And both of them apply much more to SUBMIT
than to SMTP. These days SMTP servers that act as open relays
blindly accepting mail for domains they have no direct knowledge of are
rare.

Of course configuration and implementation errors happen, so you could
end up getting mail for a remote system that ends up having a null MX.

But as you say, this case is reasonably well specified.

> But there is no such specification or

I think that to the extent there's any expectation that VRFY or EXPN would be
used for validating addresses outside the server's administrative domain,
that's an error in RFC 5321. At an absolute minimum it would warrant a
discussion of the security implications of a command that lets the client cause
a server to do lots of MX lookups and pretty much nothing else, and I don't see
any such discussion in RFC 5321.

And as far as using DNS operations of any sort as part of VRFY or EXPN support
within the administrative domain, I see that as a local matter. Really,
getting into this in at any sort of realistic level is going to require
talking about split DNS and probably various sorts of directory servers.

> but still try to open SMTP connections to the relevant target

I assume by this you mean that someone would implement null MX suppport
only in the RCPT TO code path but not in VRFY or EXPN.

If so, then I think that would require code that violates the existing MX
specification. If you treat the null MX record as somehow valid you're going to
look something up that fails to get back any A or AAAA records. You'd
have to have an implementation that falls back to A/AAAA records for the
MX itself when it sees this condition.

But regardless, since I see this as falling within the purview of 
local operation choices, I don't think it needs to be addressed.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 05 Sep 2014 11:01:55 -0700 (PDT)

> specify discussion on ietf-smtp and handle it as an individual
 
I have no problem with assigning 556 for this purpose.

But at the same time I can't resist pointing out that "corresponding" code 456
is defined in draft-martin-smtp-ipv6-to-ipv4-fallback-00.xml  as an indicator
of a need for IPv4 fallback, and should this code be assigned and that draft
adopted agents which do this correspondence thing are going to behave in
"interesting" ways.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Sep 2014 14:46:50 -0400

--On Friday, September 05, 2014 11:01 -0700 Ned Freed
<[email-masked]> wrote:

>> whether it wants to adopt/review that draft or whether I
>> should specify discussion on ietf-smtp and handle it as an
>> individual submission.
>  
> But at the same time I can't resist pointing out that
> "corresponding" code 456 is defined in
> draft-martin-smtp-ipv6-to-ipv4-fallback-00.xml  as an indicator
> of a need for IPv4 fallback, and should this code be assigned
> and that draft adopted agents which do this correspondence
> thing are going to behave in "interesting" ways.

I still read the reply code theory, as I think you mentioned
that you do, as the third digit being a discriminator but,
unlike Nyz or xNz, with no cross-code implications.   I find it
impossible to read "The third digit and any supplemental
information that may be present is reserved for the finest
gradation of information" in any other way.   

 Conversely, if one did want to follow the theory as closely as
possible, then use of an x5z code for "go fallback to something
else" seems a little off.  Maybe we should be suggesting that
draft-martin-smtp-ipv6-to-ipv4-fallback use, e.g., 541, and that
we reserve either all of 54z or a larger subset than one code
for "fallback to something else needed" or "different
capabilities needed".  That would allow an IPv6-only SMTP system
to reply with "521 IPv6 fallback necessary" when approached via
IPv4, or a NextGreatIdea system to reply with "542 Need to use
Next Big Thing, won't accept mail that narrowly conforms to 5321
or 821".   I hope I'm being a little facetious, but especially
in the wake of EAI, "if you don't ask for my preferred
extensions, I won't talk with you" isn't far away and probably
should get a distinctive code.

Grump (about the situation, not any individual or suggestion)

     john

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Sep 2014 14:53:59 -0500

On Fri, Sep 5, 2014 at 12:12 PM, Murray S. Kucherawy
<[email-masked]> wrote:
>> [...]

> something worth considering.

Right.  It's rough consensus and running code.  There's the running
code.  What it does isn't wrong, at least not for x21.

---

From: Ned Freed <[email-masked]>
Date: Fri, 05 Sep 2014 13:59:02 -0700 (PDT)

> gradation of information" in any other way.

Exactly.

> should get a distinctive code.

Not a bad idea. If that draft gets traction we should suggest making
that change.

				Ned

---

From: Kurt Andersen <[email-masked]>
Date: Thu, 4 Sep 2014 11:54:49 -0700

> mailbox name not allowed) instead.

We were wanting to use something that would be slightly distinctive so of
the two, I think 553 would be the better choice. The "not allowed" fits
with the use of a null MX although of course it is the domain rather than
the mailbox name that is the factor here.

I think that making the change is OK to do.

Cheers,
  Kurt Andersen

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 5 Sep 2014 22:20:30 -0700

> specify discussion on ietf-smtp and handle it as an individual

I was under the impression that the 521 draft would be AD-sponsored.  If
the idea of doing it through APPSAWG is back on the table, then I'm happy
to entertain a call for adoption.

Barry and Alexey, what say you?

-MSK, co-chairin' but on vacation

---

From: "John Levine" <[email-masked]>
Date: 11 Sep 2014 21:13:19 -0000

>your draft invents it, we can just refer to it in nullmx, right?

Not having heard anything for a few days, can I assume it's OK to
change the 521 to 556, and change the reference from RFC 1846, to, uh,
what?  Klensin's draft-klensin-smtp-521code-02 ?

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Thu, 11 Sep 2014 16:11:52 -0700 (PDT)

> what?  Klensin's draft-klensin-smtp-521code-02 ?

The 556 change seems fine to me.

The reference issue is beyond my pay grade.

				Ned

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 11 Sep 2014 16:37:54 -0700

> The 556 change seems fine to me.

+1.

> The reference issue is beyond my pay grade.

That reference change also looks correct (though omit the version number).

-MSK

---

From: John C Klensin <[email-masked]>
Date: Thu, 11 Sep 2014 22:09:08 -0400

--On Thursday, September 11, 2014 21:13 +0000 John Levine
<[email-masked]> wrote:

> Not having heard anything for a few days, can I assume it's OK
> to change the 521 to 556, and change the reference from RFC
> 1846, to, uh, what?  Klensin's draft-klensin-smtp-521code-02 ?

I think so, yes.   Since that draft is the only place where 556
is described, it probably has to be the document cited.   I've
now responded to the only comment I've seen about that I-D so
think it is probably ready for a Last Call as soon as some handy
AD can get around to it.  

    john

---

From: "John Levine" <[email-masked]>
Date: 13 Sep 2014 17:24:39 -0000

>That reference change also looks correct (though omit the version number).

OK, uploaded -09 with 521->556 and rfc1846 is now rfc1846bis.  Also changed
the new enhanced return code numbers to TBD per IANA suggestion.

This may introduce a dependency on 1846bis, but after eight years in the
hopper, another month or two hardly matters.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Sat, 13 Sep 2014 13:46:08 -0400

--On Saturday, September 13, 2014 17:24 +0000 John Levine
<[email-masked]> wrote:

> 
>>> > > 556 is fine with me if it's fine with everyone else.
>>> > > I'm assuming that
>>> if
>>> > Not having heard anything for a few days, can I assume
>>> > it's OK to change the 521 to 556, and change the reference
>>> > from RFC 1846, to, uh, what?  Klensin's
>>> > draft-klensin-smtp-521code-02 ?
>>> 
> OK, uploaded -09 with 521->556 and rfc1846 is now rfc1846bis.
> Also changed the new enhanced return code numbers to TBD per
> IANA suggestion.

Ok.  When you start working with the RFC Editor, be sure that
the reference points to the document whose title you quoted
(draft-klensin-smtp-521code-02).  It is not really 1846bis,
which is a separate task for the future.  Replacing 1846 would
have required addressing a series of "no mail here" issues that
are independent of the codes.  I tried doing that with
draft-klensin-rfc1846bis-00 but it because obvious in
consultation with the 1846 authors that it would be more effort
than anyone was ready for, so replaced it with
draft-klensin-smtp-521code-02.

I think the I-D is fine, just worried about possible disconnects
down the road.

> This may introduce a dependency on 1846bis, but after eight
> years in the hopper, another month or two hardly matters.

Agreed.

    john

---

From: "John Levine" <[email-masked]>
Date: 13 Sep 2014 19:02:49 -0000

>(draft-klensin-smtp-521code-02).  It is not really 1846bis,

OK, sent in a -10 with a few small twiddles to make it unambiguously
clear that's the draft in question.

R's,
John

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Sat, 13 Sep 2014 22:45:46 +0200

Hi, John,

> clear that's the draft in question.

sorry for being late at the party and excuse me if this has been 
discussed before. RFC1035 has some restrictions on labels, a.o. in par. 
2.3.1:

The labels must follow the rules for ARPANET host names.  They must
start with a letter, end with a letter or digit, and have as interior
characters only letters, digits, and hyphen.  There are also some
restrictions on the length.  Labels must be 63 characters or less.

The phrase "must start with a letter" implicitely forbids a zero-length 
label, isn't it? Furthermore, par. 3.1 of RFC1035 says:

Since every domain name ends with the null label of
the root, a domain name is terminated by a length byte of zero.

I.e. the only zero-length label is the label that terminates a series of 
labels (i.e. the root label).

/rolf

---

From: "John Levine" <[email-masked]>
Date: 14 Sep 2014 21:29:27 -0000

>labels (i.e. the root label).

Yes, that's the one that nullmx uses.

I don't understand what you think is wrong here.

R's,
John

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 15 Sep 2014 21:55:31 +0200

> I don't understand what you think is wrong here.

nothing wrong, the zero-length label and the single dot were a bit 
confusing, but after re-reading the draft and RFC1035 it's now clear to 
me that draft and RFC1035 are coherent ;-).

/rolf